Stealthy Shield Defense: A Conditional Mutual Information-Based Approach against Black-Box Model Inversion Attacks
Authors: Tianqu Zhuang
Hongyao Yu
Yixiang Qiu
Hao Fang
Bin Chen
Shu-Tao Xia
Venue: ICLR 2025 Poster (2025)
Keywords: AI security
model inversion attack
information bottleneck
conditional mutual information
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: Resist black-box model inversion attacks via post-processing.

Abstract: Model inversion attacks (MIAs) aim to reconstruct the private training data by accessing the public model, raising concerns about privacy leakage. Black-box MIAs, where attackers can only query the model and obtain outputs, are closer to real-world scenarios. The latest black-box attacks have outperformed state-of-the-art white-box attacks, and existing defenses cannot resist them effectively. To fill this gap, we propose Stealthy Shield Defense (SSD), a post-processing algorithm against black-box MIAs. Our idea is to modify the model's outputs to minimize the conditional mutual information (CMI). We mathematically prove that CMI is a special case of Information Bottleneck (IB), and thus inherits the benefits of IB---making predictions less dependent on inputs and more dependent on ground truths. This theoretically guarantees our effectiveness, both in resisting MIAs and preserving utility. To minimize CMI, we formulate a convex optimization problem and solve it via the water-filling method. Without the need to retrain the model, our defense is plug-and-play and easy to deploy. Experimental results indicate that SSD outperforms existing defenses, in terms of MIA resistance and model's utility, across various attack algorithms, private datasets, and model architectures. Our code is available at https://github.com/ZhuangQu/Stealthy-Shield-Defense.

Published as a conference paper at ICLR 2025
STEALTHY SHIELD DEFENSE: A CONDITIONAL MUTUAL INFORMATION-BASER APPROACH AGAINST BLACK-BOX MODEL INVERSION ATTACKS
Tianqu Zhuang", Hongyao Yu" Yixiang Qiu Hao Fang",, Bin Chen Shu-Tao Xia 'Shenzhen International Graduate School, Tsinghua University, China
PHarbin Institute of Technology. Shenzhen, China
) (zhuangtq23, qiu-yx24, fang-h22 h23) @ maiils. maiilstsinnguu.eduu yuhongyao wtw.asitsttuuuuu...... chenbin2021 1 hi;,.edu.cn; xiast @ sz. www.tsinnhua..u...oo 'Equal contribution "Corresponding author
ABSTRACT
Model inversion attacks (MIAs) aim to reconstruct the private training data by accessing the public model, raising concerns about privacy leakage. Black- box MIAs, where attackers can only query the model and obtain outputs, are closer to real-world scenarios. The latest black-box attacks have outperformed state-of-the-art white-box attacks, and existing defenses cannot resist them ef- fectively. To fill this gap, we propose Stealthy Shield Defense (SSD), post- processing algorithm against black-box MIAs. Our idea is modify the model's outputs to minimize the conditional mutual information (CMI). We mathemat- ically prove that CMI a special case of Information Bottleneck (IB), and thus inherits the benefits of IB IB-making predictions less dependent on inputs and more dependent on ground truths. This theoretically guarantees our ef- fectiveness, both in resisting MIAs and preserving utility. To minimize CMI, we formulate a convex optimization problem and solve via the water-filling method. Without the need to retrain the model, our defense is plug-and-play and easy to deploy. Experimental results indicate that SSD outperforms exist- ing defenses, in terms of MIA resistance and model's utility, across various attack algorithms, private datasets, and model architectures. Our code available at https: /tttps://giie /ttp:////gttuuuuueee thu.. com http///hhaaaauu/aaae/aa Stea 1 Chy-Shieed-Dae Defenssse
INTRODUCTION
Deep neural networks (DNNs) have driven widespread deployment in multiple mission-critical do- mains, such as computer vision (He et al. 2015), natural language processing (Devlin et al.] 2019) and dataset distillation (Zhong al. 2024b ab. However, their integration with sensitive training data has raised concerns about privacy breaches. Recent studies (Fang et al., 2024b a 2025) have explored various attack methods to probe such privacy, such gradient inversion (Fang al. 2023 Yu et al.) 20245) and membership inference (Hu et al.) 2021). Among the emergent threats, model inversion attacks (MIAs) aim reconstruct the private training data by accessing the public model, posing the greatest risk (Qiu et al. (022bb.. For instance, consider a face recognition access control system with publicly accessible interface. Through carefully crafted malicious queries, model in- version attackers can infer the sensitive facial images stored in the system, along with the associated user identities.
MIAs are divided into white-box and black-box (Fang et al. 2024c). White-box attackers know the details of the model, whereas black-box attackers can only query the model and obtain outputs. Black-box MIAS become more threatening than white-box because: (1) Black-box scenarios are more common. As models grow larger nowadays, they are mostly stored on servers and can only be accessed online, which is typical black-box scenario. (2) Black-bov attacks are more powerful. The latest soft-label attack RLBMI (Han et al. 2023) and hard-label attack LOKT (Nguyen et al. 2023) have outperformed state-of-the-an the- white-boo box attacks. (3) Existing defenses cannot resist black-box attacks effectively. Existing defenses focus on modifying the weights and structure of the model, but black-box attackers only exploit the outputs, and thus are less susceptible.

---

Published as a conference paper at ICLR 2025
Alice Bob
Chris
Distributions of Y Post-Processing Distributions of Y with High CMI
Ground Truth Labels Y
Inputs
Target Model
Algorithm
with Low CMI
(x; P)
((:
Figure 1: An overview of Stealthy Shield Defense. With 3 classes, the probability simplex a triangle. CMI defined as Z(X; ÝIY). According to our Theorem | minimizing CMI makes the mutual information I(X; Y) minimized and I(Ý;Y) maximized. As shown by Yang et al. 20024). minimizing CMI makes the outputs more concentrated class-wisely.
To address these concerns, we propose Stealthy Shield Defense (SSD), post-processing algorithm against black-box MIAs. As shown in Figure ! the idea of SSD is to modify the model's outputs to minimize the conditional mutual information (CMI). CMI quantifies the dependence between inputs and predictions when ground truths are given. In Theorem [ we prove that CMI is special case of Information Bottleneck (IB), and thus inherits the benefits of IB IB-making predictions less dependent on inputs and more dependent ground truths. Under this theoretical guarantee, SSD achieves a better trade-off between MIA resistance and model's 's utility. Without the need to retrain the model, SSD is plug-and and-play and easy deploy.
The contributions of this paper are:
We introduce CMI into model inversion defense for the first time, and theoretically prove its effectiveness.
We propose a post-processing algorithm to minimize CMI without retraining models. In our algorithm, temperature is introduced to calibrate the sampling probability and adaptive dist--istortion introduced to constrain the modification to outputs. We speed up our algorithm by GPU-based water-filling method well.
Our experiments indicate that we outperform competitors, terms of MIA resistance and model's 's utility, exhibiting strong generalizability across various attack algorithms, pri- vate datasets, and model architectures.
2 RELATED WORKS
2.1 MODEL. INVERSION ATTACKS AND DEFENSES
Model inversion attacks (MIAs) a serious privacy threat to released models (Fang et 2024c)
MIAS are categorized as white-bon box Zhang et al. 2019 Chen et al. 2020 Struppek et al. 2022
Yuan et al. 2023 Qiu et al. 2024a) and black-box. We focus on black-box MIAs, where attackers
can only query the model and obtain outputs. In this scenario, BREP (Kahla et al. (022) utilizes
zero-order optimization to drive
latent vectors away from the decision boundary. Mirror (An
et al. (022) and C2FMI (Ye et al.) (023) explore genetic algorithms. LOKT (Nguyen et al.
2023)
trains multiple surrogate models and applies white-box attacks to them.
To address the threat of MIAs, variety of defenses have been proposed. MID (Wang al.) 2020 BiDO (Peng et al., (022), and LS (Struppek et al.) (023) change the training losses, TL (Ho et al.
2024h freezes some layers of the model, and CA-FaCe (Yu et al. 2024a) changes the structure of the model. However, black-box attackers only exploit the outputs, and thus are rarely hindered. The defense against black-box MIAs is still limited.
In this paper, we propose novel black- box defense based on post post-processing, without retraining the model. Experimental results indicate that we outperform these existing defenses.
2

---

Published as a conference paper at ICLR 2025
2.2 INFORMATION BOTTLENECK AND CONDITIONAL MUTUAL INFORMATION
Tishby et al. (2000) proposed the Information Bottleneck (IB) principle: a good machine learning model should compress the redundant information in inputs while preserving the useful information for tasks. They later highlighted that information is compressed layer-by-layer in DNNs Tishby & Zaslavsky 2015 Shwartz-ZN -iiv & Tishby 2017). Alemi al. (2017) proposed Variational Infor- mation Bottleneck (VIB) to estimate the bounds of IB, and Wang ct al. 2020) applied VIB to their Defense (MID).
Mutual Information-
Yang et al. )2024) proposed to use conditional mutual information (CMI) as performance metric for DNNs, providing the calculation formula and geometric interpretation of CMI. By minimizing CMI, they improve classifiers (Yang et al. 2025) and address class imbalance (Hamidi et al. 2024). By maximizing CMI, they improve knowledge distillation (Ye et al. 2024) and address nasty teachers Yang & Ye 2024).
In this paper, we theoretically prove that CMI is special case of IB and thus inherits the benefits of IB. Furthermore, we propose novel model inversion defense based CMI.
3 PRELIMINARY
3.1 NOTATIONS
Let f: X + W be neural classifier, X X be input, Y Y be the ground truth label,
Ý E Y be the label predicted by and Z E be the intermediate representation in Note that Z + Y is Markov chain. Let P be the probability function and P(x) P(X = z).
Y
Pvy) = P(Y = v}. P(x, ülw) == P(X = x, Y = 7 Y = y). etc.
Let A be the probability simplex over Y. fxx E A be the output from the softmax layer when is input to f. and fa(x) E (0,1) ) be the ğ-th component of fzx). 1 Y.
3.2 MODEL INVERSION ATTACKS
Let D C X x Y be the dataset learned by f. MIAs aim reconstruct Ó as close to D as possible. Based on the access to f, MIAs are categorized as:
Hard-label: Attackers can query any x E X and obtain f(x) E Y. i.e. argmax fz().
Soft-label: Attackers query any E X and obtain f(x E A DEY
White-box: Attackers know the details of f.
Hard-label and soft-label, collectively called black-box Boxxparee are defended against in this paper.
3.3 MUTUAL INFORMATION-BASSI BASED DEFENSE (MID)
Wang et al. (2020) proposed to resist MIAs by reducing the dependence between X and Ý. The dependence quantified by the mutual information, which is defined as
P(x, 0)
I(X:)= = P(x, log P(z)P(0)
ZEX DEY
They reduced I(X; Y) prevent attackers from inferring the information about D. However, low I(X; Y) hurts the model's utility. Especially, T(X; 0 iff X and y are independent, in which case f is immune to any attack but useless at all.
As an alternative, they introduced the Information Bottleneck Tishby & Zaslavsky 2015). which is defined as
(( -
where 8 > 0. They used (2 as regularizer to train f. minimizing I(X; Z) to resist MIAs while maximizing I(Z;Y) preserve the model's utility.
Some literature refers hard-label lahel-only, and soft-label black-box.
3

---

Published as a conference paper at ICLR 2025
4 METHODOLOGY
4.1 CONDITIONAL MUTUAL INFORMATION-BANE BASED DEFENSE
We aim resist black-box MIAs where attackers cannot access Z, so we still minimize I(X; P) instead of (X;;; Z). Furthermore, we observe that all MIA algorithms target one fixed label during attacking. Formally, let
D= E X: (z. v) E D}
be the sub-dataset labeled with the ground truth y. When y is given, all attackers aim to reconstruct Du as close D as possible. Against their intention, we propose to minimize
P(x, klv)
I(X;Y1Y = v)
ΣΣ DEXSEY
Par log P(rly))(Ul))
(3)
I(X;YIY = 3) quantifies the dependence between X and Y when Y = We minimize 3 to
prevent attackers from inferring the information about D To protect the complete D, we minimize for each y E W with the weight of Ptr). This is equivalent to minimizing the conditional mutual information (CMI), which is defined as
I(x; VIV) = P(())- -(;-V =
(4)
DEY
Theorem 1, CMI a special case of Information Bottleneck (2) taking Z = Ý and B 1, i.e.
I(x;Yl)) = I(x;))) - I(.;Y).
Our proof is in Appendix A Our theorem proves that CMI inherits the benefits of Information Bottleneck (IB), i.c., minimizing CMI has two effects:
Minimize I (X; Ý) to compress the redundant information in inputs, and decrease the de- pendence between inputs and predictions. This improves the MIA resistance shown by Wang et al. (202)).
(X;
Maximize I(Ý;Y Y) preserve the useful information for tasks, and increase the depen- dence between predictions and ground truths. This improves the model's utility obviously.
I(X; Z) in IB is challenging calculate because X and E both high-dimensional. Wang et al. 2020) could only approximate IB by variational bounds (Alemi et al. 2017). Fortunately. as special case of IB, CMI can be calculated directly (Yang et 2022).
4.2 MINIMIZE CMI VIA POST-PROCSSIING
Previous works minimized CMI during training (Yang et al. 2024 Hamidi et al., 2024; Yang et al, 2025 In contrast to them, we propose a training-free free method to minimize CMI. We have
(ar,,
((X;Y1)) Σ P(u) Σ Σ P(x, Bly) log P(xl)))((lv))
by definitions B
NEY
ZEX DEY
P(jlary). P(glv)
ΣΣΣ
P(x, 9. y) log
ZEX JEY DEY
P(ile.v)
Σ P(z)
P(y|z)
P(j 2. v) log
P(ūly)
DEY
P(j]x)
Σ xEX
P(x)
Ply Σ P(jlz) log
by Markov chain Y + - V.
P(ol))'
WEY
SEY
Thus minimizing ((X;İlY) is equivalent to minimizing ΣyeV P(y|z) Σ Sev P(ū]z) log P(ilz) for each input to f. For simplicity, we sample y E V with the probability of P(y(a)) and minimize
P(
4

---

Published as a conference paper at ICLR 2025
Eger 2()(2) log P(yl)) instead which equals the original objective in terms of expectation. Next we need to calculate z) ()) and P()le).
P()lz)
To a( we have P() (x) by design of neural classifiers.
To get P(y(z), an intuitive idea is that P(ylar) P())=) y = v, Guo al. (2017) had demonstrated its inaccuracy in modern neural classifiers. Inspired by their work, we introduce the temperature to calibrate it.
To get P(jly), we have
p(
EPC,,00)) - E((xy))(()E Σ P(xl)))(())) xEX
aEX
aEX
Σ P(xl))ff()) = Ex Y = mEX
mean fo(x"), z'nn
where R is by that the samples in D are i..d. to p(r and thus the sample mean can estimate the conditional expectation. In practice we use the validation D because the training samples are overfitted by f and may cause inaccurate estimation.
Now our objective becomes
P((lz) P(ll))
fa(z) fo(x*)
P(( log
Σ DEY
ff(x) log
(L(f(x)) mean f(z)),
(5)
mean r'
DEY
where KL is Kullback-Leibler divergence. To minimize (5), we fix mean (x²" for simplicity and w'EDV
modify fxx). Letp A be the modified version of f(x) and our objective is KL mean f (xx)).
æ'EDV
Additionally, we constrain |pp - f(x)]I I to preserve the model's utility, where > is the
distortion controller.
In information theory, minimizing the mutual information under bounded distortion constraints is known as the rate-distortion problem (Shannon 1959), which is for signal compression. If a signal has less information, it is easier to compress, and stricter distortion bound can be applied. Inspired by his work, we introduce Shannon entropy to quantify the information in Ý when X = which is defined as
H(`IX = x) =- Σ DeY
P(j]z) log P(jlz).
Our constraint becomes | S H where the distortion bound is proportional to the amount of information. This new constraint reduces the modification when the information limited, and enhances the compression when the information is abundant. We refer this as adaptive rate distoriion....
Our defense is summarized Algorithm I plug-and- and-play and easy to deploy.
Without the need retrain the model, our defense is
Note that D' {q" y Y) can be calculated and stored in advance. If the model owner and the defender are not the same, the owner only needs to provide D' instead of D, avoiding communication costs and privacy risks.
(6) can be solved by existing convex optimizers. Furthermore, we derive the explicit solution in Appendix B calculate it within O (W) log [Y] time in Algorithm accelerate it via GPUs in Algo- rithm and evaluate the computational cost in Appendix
With sampling. we only need to consider one E Y. so we can solve 6 within O( log [Y]) time by Algorithm 2 Without sampling, have to consider all y E W. The problem complexity is ?)) which is unacceptahle when V is large.

---

Published as a conference paper at ICLR 2025
Algorithm 1: poott-proesssin to minimize CMI.
Input:
original output f(z), validation set D. temperature T, distortion controller E.
Output: modified output p".
Sample y Y with the probability of softmaxx f(2) q² mean (=²):
x² 1
H - Σ [5( log (x):
BEY
Solve the convex optimization problem:
min (L(pll9"),
1p f(x)ll s H. PEA
(6)
return the optimal solution p":
5 EXPERIMENTS
5.1 SETTINGS
Datasets. We select CelebA (Liu et al. 2015) and FaceScrub (Ng & Winkler 2014) the private datasets, and FFHQ (Karras et al. 2018) as the public dataset. CelebA has 10,177 labels and we take 1,000 labels with the most images. FaceScrub has 530 labels and 106,863 images, but we only take 43,147 images because the other URLs are broken. FFHQ has 70,000 unlabeled images. All images cropped and resized to 64 64. We use 80% of the private samples for training, 10% for validation, and 10% for testing.
2015) and VGG-16 (Simonyan & Zisserman 2014 as the Models. We select IR-152 (He et al.
2022) as the evaluation models. IR-152 and Max VTT are pre- target models, and Max (Tu et al. trained on MS-Celeb- 1M (Guo et al. 2016), and VGG-16 is pre-trained
ImageNet (Deng al., 2009). They are fine-tuned on the training set, and we select the best version by the validation set. The evaluation model on CelebA achieves 97.3% test accuracy, and the one FaceScrub achieves 99.3%.
Attacks. We select Mirror (An et 2022). C2FMI Ye al. 2023 and RLBMI (Han 2023 as the soft-label attackers, and BREP (Kahla et al. 2022 and LOKT (guyynn et al. 2023) as the hard-label attackers. They attack the first 100 private labels and reconstruct images per label. For RLBMI, BREP and LOK'T, I, we train GANs and surrogate models on FFHQ. For Mirror and C2FMI, we use the 256 x 256 StyleGAN2 (Karras al. 2019) trained on FFHQ, whose generated images are center-cropped to 176 x 176 and resized to 64 x 64.
Defenses. We select MID WWag et al. 2020). BiDO (Peng et al. 2022). LS (Struppek et al. 2023) TI. (Ho et al.] (024) and Purifier (Yang et al. 2023) as the competitors. Purifier trains CVAE, and the others retrain the target models. We carefully tune their
to achieve the similar validation accuracies. All perrme-ar of defenses are in Appendix D
To evaluate the MIA resistance and model's utility, we consider the following metrics:
Attack Accuracy Let the evaluation model reclassify the reconstructed images. The top-1 and top-5 accuracies are denoted Accl and Acc5 respectively. Lower percentages indicate better MIA resistance.
Feature Distance
The image features extracted from the penultimate layer of model. We take the average L2 feature distance between the reconstructed images and the nearest training images. The features extracted by the evaluation model and FaceNet (Schroff et al.) 2015) trained on VGGFace2 (Cao et al. (017). denoted as S and &face respectively. Higher distances indicate better MIA resistance.
Test Accuracy The accuracy of the target model on the test set, denoted as Acc. Higher percentage indicates better utility.

---

Published as a conference paper at ICLR 2025
Distortion The L1 distance between the outputs with and without defense. We take the average the test set, denoted as Dist. Lower distance indicates better utility.
All experiments are conducted on MIBench (Qiu et al. 2024b). The experiments about RLBMI, Purifier and high resolution Appendix G
Table MIA resistance of various defenses under soft-label attacks.
Mirror
C2FMI
1 Accl
Acc5
Seval
1 6gace
Accl
1 Ace5
1 deval
1 tace
None
18.0%
31.0%
625
1.22
5.8%
15.4%
647
1.30
MID
17.8%
31.6%
629
1.22
1.2%
4.2%
1.48
IR-152 ColebA
BiDO
10.6%
25.8%
614
1.16
5.4%
13.0%
645
1.33
LS
9.0%
15.2%
660
1.36
1.8%
8.0%
660
1.36
TL
13.6%
28.6%
633
1.24
2.8%
5.0%
703
1.48
SSD
3.2%
8.2%
728
1.46
1.2%
1.8%
744
1.59
None 55.2%
76.8%
496
0.89
19.6%
41.4%
549
1.02
FaceScrub
MID
38.0%
61.0%
534
0.96
0.2%
1.8%
715
1.44
IR-152
BiDO
34.4%
60.4%
526
0.93
13.2%
27.6%
588
1.13
LS
45.8%
73.4%
503
0.90
13.6%
31.2%
564
1.05
TL
39.2%
63.2%
535
0.99
7.0%
18.6%
617
1.22
SSD
32.2%
46.4% %
604
1.15
1.6%
5.6%
671
1.35
None 29.0%
51.8%
544
1.00
12.8%
30.6%
588
1.11
VGG- 16 FaceScrub
MID
34.6%
64.8%
520
0.94
4.4%
17.8%
611
1.19
BiDO
20.0%
44.4%
556
1.03
12.0%
27.6%
575
1.09
LS
30.2%
56.4%
531
0.97
15.6%
37.4%
551
1.05
TL
17.8%
41.8%
558
1.03
8.4%
27.2%
574
1.10
SSD
16.4%
40.0%
604
1.13
3.2%
13.6%
656
1.29
Table 2: MIA resistance of various defenses under hard-label attacks.
BREP
LOKT
4 Acc1
4 Acc5
Serat
t Ofac
Accl
4 Acc5
t deval
t 8facc
None
26.8%
50.8%
541
1.02
50.2%
67.8%
527
0.93
MID
20.8%
39.4%
596
1.13
32.4%
52.2%
551
1.00
IR- 122 CclebA
BiDO
11.4%
33.4%
577
1.09
30.8%
48.4%
576
1.06
LS
19.4%
43.8%
547
1.03
33.6%
57.0%
537
1.01
TL
20.2%
46.4%
550
1.05
43.0%
66.2%
525
0.93
SSD
1.8%
4.2%
769
1.62
1.2%
9.0%
769
1.61
None 50.6%
77.2%
497
0.95
90.2%
96.2%
411
0.65
IR- IR-152 FaceScrub
MID
36.2%
52.6%
570
1.10
61.8%
81.4%
475
0.78
BiDO
33.6%
62.6%
530
1.02
82.2%
94.8%
445
0.69
LS
36.0%
67.0%
521
1.01
84.8%
95.4%
429
0.69
TL.
30.4%
59.0%
538
1.06
91.6%
98.4%
414
0.65
SSD
9.2%
13.4%
717
1.47
15.8%
25.0%
656
1.27
None 28.2%
54.0%
543
1.06
75.8%
91.4%
444
0.74
VGG-16 FaceScrub
MID
30.0%
56.8%
535
1.03
63.2%
86.8%
470
0.80
BiDO
21.6%
44.4%
566
1.10
67.4%
87.4%
474
0.80
LS
24.0%
50.4%
549
1.05
74.4%
90.6%
453
0.79
TL
20.8%
46.8%
566
1.08
67.4%
88.8%
471
0.79
SSD
10.8%
18.6%
693
1.40
24.0%
35.4%
635
1.24

---

Published as a conference paper at ICLR 2025
5.2 COMPARISONS WITH -TATE-OF-THE-AA ART DEFENSES
Table Tshows the evaluation results under soft-label attacks, and Table the ones under hard- label. Our SSD exhibits the best MIA resistance and outperforms all state-of-the-an defenses, across various attack algorithms, private datasets, and model architectures. This is because we minimize the CMI to prevent attackers from inferring the information about the private dataset.
Surprisingly, some defenses perform worse than no defense in some indicators. demonstrates that existing defenses may fail to resist modern black-box attacks. They focus modifying the weights and structure of the model, but black-bos box attackers only exploit the outputs and thus are rarely hindered. This highlights the necessity of designing specialized black-box defenses.
Figure shows the reconstructed images from IR-152 models trained on CelebA. Our SSD signif- icantly enhances the difference between D and DV, forces attackers to reconstruct wrong images, and thus protects the private data. In contrast, other defenses fail to prevent attackers from recon- structing similar images, leading to privacy leakage.
Label
Dataset
Attack
Mirror C2FMI
BREP
LOKT
Mirror C2FMI
BREP LOKT
Mirror C2FMI
BREP
LOKT
Defense
None
MID
BIDO
LS
SSD
Figure 2: The reconstructed images from various attacks facing various defenses. Above are the ground truth labels y and the private sub-datasets DV (4 images shown). Below are the reconstructed datasets DV (1 image shown).
Table 3: Model's utility with various defenses.
IR-152 & CelebA
IR-152 & FaceScrub
VGG-16 & FaceScrub
Acc
Dist
Acc
$ Dist
Acc
1 Dist
None
92.1%
0
98.2%
0
91.8%
0
MID
86.8%
0.607
95.5%
0.321
86.4%
0.744
BiDO
86.6%
0.371
95.3%
0.135
88.5%
0.313
LS
86.9%
0.317
95.6%
0.103
87.5%
0.295
TL
86.5%
0.352
95.8%
0.128
87.4%
0.306
SSD
87.1 %
0.191
97.0%
0.055
89.3%
0.176
8

---

Published as a conference paper at ICLR 2025
Table 3 shows the evaluation results model's utility. Our SSD achieves the highest test accuracy, because we minimize CMI to preserve the information useful for tasks. Additionally, SSD holds the lowest distortion because of our adaptive rate-distortion constraint.
5.3 VISUALIZE THE OUTPUTS
visualize how SSD modifies the outputs. The target model is a VGG-16 16 model trained on FaceScrub. shows that minimizing CMI makes the outputs aggregate into clusters, as demonstrated by Yang al. (2024). For private samples, this reduces the sensitivity between inputs and outputs, preventing attackers from inferring the private information. For attack samples, this misleads attackers towards the wrong label.
We use t-SNE van der Maaten & Hinton 2008)
(a) Tested w/o defense.
(b) Tested with SSD.
(c) Attacked w/o defense. (d) Attacked with SSD.
Figure 3:
Visualizing the outputs by t-SNE. For (a) and (b), the test samples are colored according their ground truth labels. For (c) and (d), the attack samples generated by Mirror.
5.4
ABLATION EXPERIMENTS
Attack Accuracy
60.0
%
Accl
6
1.60
Attack Accuracy
60.0
Acc1
Sone
n
Sace W/o Ada.
1.60
Face Distance
Face Distance
45.0
1.40
45.0
1.40
1.20
15.0
1.00
15.0
1.00
0.80
0,80
0.04
0.10 0:13
0.2
0.5
(a) Temperature
(c) Distortion Controller
0.12
Test Accuracy
Test Accuracy
98.0
Distortion
98.0
Distortion
0.08
97.5
97.5
0.04
0.04
96.5
96.5
w/o T. 0.01 0.04 0.07 0.10 0.13 0.16 0.19 (b) Temperature
0.1
0.2
0.3
0.4
0.5
0.6 0.7
0.8
(d) Distortion Controller
Figure 4: The ablation results. The attacker of (a) is BREP, and (c) is Mirror. "w/o T." denotes "without temperature", and "w/o Ada." denotes "without adaptive rate-distortion".
To explore the effects of temperature and distortion controller E, we conduct ablation experiments on IR-152 models trained on FaceScrub.

---

Published as a conference paper at ICLR 2025
Figure 4 a) shows that higher temperature helps to resist hard-label attacks, because it makes the sampling probability more uniform, and attackers easier to get misleading labels. However, a higher temperature decreases the test accuracy, shown in Figure 4b).. Specifically, without the temperature, neither the MIA resistance nor model's accuracy is satisfactory, which highlights the necessity of temperature.
Figure c) shows that a larger distortion bound helps to resist soft-label attacks, because it allows greater modifications. However, larger distortion bound decreases the test accuracy and increases the distortion, as shown in Figure [].. Specifically, without the adaptive rate-distortion, neither the face distance nor the distortion is satisfactory, which highlights the necessity of adaption.
CONCLUSION
In contrast to previous researches model inversion defense with a focus on white-box attacks, we conduct a specific study on black-box attacks. Specifically, we investigate the impact of conditional mutual information (CMl) and develop CMI-based defense strategy. We conduct our defense the post-processing stage instead re-training model. To further reduce the modifications to outputs, we introduce adaptive rate-distortion framework and optimize it by the water-filling method. Experimental results demonstrate that our defense method achieves state-of-the-ar (SOTA) performance against black-box attacks. We hope that findings will help shift attention toward robust defense mechanisms in black-box settings and inspire further researches in this area.
7 ACKNOWLEDGEMENT
This work is supported by National Natural Science Foundation of China (62171248, 12301189), Peng Cheng Laboratory (PCL2023A08), and Shenzhen Science and Technol- ogy Program (KIZD220200931037020000 0CC000040001Y0022200111111101110 1CBS20221008003 CXWD200008111123600,,
REFERENCES
Alexander A. Alemi, Ian Fischer, and Joshua V. Dillon. Deep variational information bottleneck. International Conference Learning Representations (ICLR), 2017.
Shengwei An, Guanhong Tao, Qiuling Xu, Yingqi Liu, Guangyu Shen, Yuan Yao, Jingwei Xu, and X. Zhang. Mirror: Model inversion for deep learning network with high fidelity. Network and Distributed System Security Symposium (NDSS). 2022.
Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In Automatic Face & Gesture Recognition (FG), 2017.
Si Chen, Mostafa Kahla, R. Jia, and Guo-Jun Qi. Knowledge-enriched distributional model inversion attacks. International Conference on Computer Vision (ICCV), 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, K. Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition (CVPR), 2009.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. North American Chapter of the Associa- tion for Computational Linguistics (NAACL), 2019.
Hao Fang, Bin Chen, Xuan Wang, Zhi Wang, and Shutao Xia. Gifd: generative gradient inversion method with feature domain optimization. International Conference on Computer Vision (ICCV), 2023.
Hao Fang, Jiawei Kong, Bin Chen, Tao Dai, Hao Wu, and Shutao Xia. Clip-guided generative networks for transferable targeted adversarial attacks. European Conference on Computer Vision (ECCV), 2024a.
10

---

Published as a conference paper at ICLR 2025
Hao Fang, Jiawei Kong, Wenbo Yu, Bin Chen, Jiawei Li, Shutao Xia, and Ke Xu. One perturbation is enough: On generating universal adversarial perturbations against vision-lanne -lanuale pre-training models. ArXiv, 2024b.
Hao Fang, Yixiang Qiu, Hongyao Yu, Wenbo Yu, Jiawei Kong, Baoli Chong, Bin Chen, Xuan Wang, and Shutao Xia. Privacy leakage dnns: A survey of model inversion attacks and defenses. ArXiv, 2024c.
Hao Fang, Xiaohang Sui, Hongyao Yu, Jiawei Kong, Sijin Yu, Bin Chen, Hao Wu, and Shutao Xia. Retrievals can be detrimental: A contrastive backdoor attack paradigm on cetrieval-augmente diffusion models. ArXiv, 2025.
A. Gionis, Piotr Indyk, and Rajeev Motwani. Similarity search in high dimensions via hashing. In Very Large Data Bases Conference (VLDB), 1999.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. International Conference on Machine Learning (ICML), 2017.
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb-Im: A dataset and benchmark for large-scale face recognition. In European Conference on Computer Vision (ECCV), 2016.
Shayan Mohajer Hamidi, Renhao Tan, Linfeng Ye, and En-Hui Yang. Fed-it: Addressing class imbalance in federated learning through information- theoretic lens. International Symposium on Information Theory (ISIT), 2024.
Gyojin Han, Jachyun Choi, Haeil Lee, and Junmo Kim. Reinforcement learning-based black-box model inversion attacks. Computer Vision and Pattern Recognition (CVPR), 2023.
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Computer Vision and Pattern Recognition (CVPR), 2015.
Sy-Tuyen Ho, Koh Jun Hao, Keshigeyan Chandrasegaran, Ngoc-Bao Nguyen, and Ngai-Man Che- ung. Model inversion robustness: Can transfer learning help? Computer Vision and Pattern Recognition (CVPR), 2024.
Hongsheng Hu, Zoran A. Salcic, Lichao Sun, Gillian Dobbie, P. Yu, and Xuyun Zhang. Membership inference attacks on machine learning: survey, ACM Computing Surveys (CSUR), 2021.
Mostafa Kahla, Si Chen, Hoang A. Just, and R. Jia. Label-only model inversion attacks via boundary repulsion. Computer Vision and Pattern Recognition (CVPR), 2022.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. Computer Vision and Pattern Recognition (CVPR), 2018
Tero Karras, Samuli Laine, Mika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Ana- lyzing and improving the image quality of stylegan. Computer Vision and Pattern Recognition (CVPR), 2019.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. International Conference Computer Vision (ICCV), 2015.
Hongwei Ng and Stefan Winkler. data-driven approach to cleaning large face datasets. Interna- lional Conference on Information Photonics (ICIP), 2014.
Ngoc-Bao Bao Nguyen, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Cheung- Label-only model inversion attacks via knowledge transfer. Neural Information Processing Sys- tems (NeurIPS), 2023.
Xiong Peng, Feng Liu, Jingfeng Zhang, Long Lan, Junjie Ye, Tongliang Liu, and Bo Han. Bilateral dependency optimization: Defending against model-innersionn attacks. Knowledge Discovery and Data Mining (KDD), 2022.
11

---

Published as a conference paper at ICLR 2025
Yixiang Qiu, Hao Fang, Hongyao Yu, Bin Chen, Meikang Qiu, and Shutao Xia. A closer look at gan priors: Exploiting intermediate features for enhanced model inversion attacks. European Conference on Computer Vision (ECCV), 2024a.
Yixiang Qiu, Hongyao Yu, Hao Fang, Wenbo Yu, Bin Chen, Xuan Wang, Shutao Xia, and Ke Xu. Mibench: A comprehensive benchmark for model inversion attack and defense. ArXiv, 2024b.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. Computer Vision and Pattern Recognition (CVPR), 2015.
Claude Elwood Shannon. Coding theorems for a discrete source with fidelity criteria. Ire National Convention Record, 1959.
Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via informa- tion. ArXiv, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogmition. International Conference on Learning Representations (ICLR), 2014.
Lukas Struppek, Dominik Hintersdorf, Antonio De Almeida Correia, Antonia Adler, and Kristian Kersting. Plug play attacks: Towards robust and flexible model inversion attacks. International Conference on Machine Learning (ICML), 2022.
Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting. Be careful what you smooth for: Label smoothing can be privacy shield but also a catalyst for model inversion attacks. International Conference on Learning Representations (ICLR), 2023.
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. Infor- mation Theory Workshop (ITW), 2015.
Naftali Tishby, Fernando C Percira, and William Bialek. The information bottleneck method. ArXiv, 2000.
Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Conrad Bovik, and Yinxiao Li. Maxvit: Multi-axis vision transformer. In European Conference on Computer Vision (ECCV), 2022.
Laurens van der Maaten and Geoffrey E. Hinton. Visualizing data using t-sne. Journal of Machine Learning Research (JMLR), 2008.
Tianhao Wang, Yuheng Zhang, and R. Jia. Improving robustness to model inversion attacks via mutual information regularization. AAAI Conference on Artificial Intelligence (AAAT), 2020.
En-Hui Yang and Linfeng Ye. Markov knowledge distillation: Make nasty teachers trained by self- undermining knowledge distillation fully distillable. European Conference on Computer Vision (ECCV), 2024.
En-Hui Yang, Shayan Mohajer Hamidi, Linfeng Ye, Renhao Tan, and Beverly Yang. Conditional mutual information constrained deep learning: Framework and preliminary results. International Symposium on Information Theory (ISIT), 2024.
En-Hui Yang, Shayan Mohajer Hamidi, Linfeng Ye, Renhao Tan, and Beverly Yang. Conditional mutual information constrained deep learning for classification. IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2025.
Ziqi Yang, Li-Juan Wang, D. Yang, Jie Wan, Ziming Zhao, Ee-Chien Chang, Fan Zhang, and Kui Ren. Purifier: Defending data inference attacks via transforming confidence scores. AAAI Con- ference on Artificial Intelligence (AAAI), 2023.
Linfeng Ye, Shayan Mohajer Hamidi, Renhao Tan, and En-Hui Yang. Bayes conditional distribution estimation for knowledge distillation based on conditional mutual information. International Conference Learning Representations (ICIR), 2024.
12

---

Published as a conference paper at ICLR 2025
Zipeng Ye, Wenjian Luo, Muhammad Luqman Naseem, Xiangkai Yang, Yuhui Shi, and Yan Jia. C2fmi: Corse-to-fine black-box model inversion attack. IEEE Transactions on Dependable and Secure Computing (TDSC), 2023.
Hongyao Yu, Yixiang Qiu, Hao Fang, Bin Chen, Sijin Yu, Bin Wang, Shutao Xia, and Ke Xu. Calor: Towards comprehensive model inversion defense. ArXiv, 2024a.
Wenbo Yu, Hao Fang. Bin Chen, Xiaohang Sui, Chuan Chen, Hao Wu, Shutao Xia, and Ke Xu. Gi-nas: Boosting gradient inversion attacks through adaptive neural architecture search. ArXiv, 2024b.
Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Neng H. Yu, and Yangyi Zhang. Pseudo label-guided model inversion attack via conditional generative adversarial network. AAAI Con- ference Artificial Intelligence (AAAI), 2023.
Yuheng Zhang, Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Xiaodong Song. The secret revealer: Generative model-inversion attacks against deep neural networks. Computer Vision and Pattern Recognition (CVPR), 2019.
Xinhao Zhong, Bin Chen, Hao Fang. Xulin Gu, Shutao Xia, and En-Hui Yang. Going beyond feature similarity: Effective dataset distillation based on class-aware conditional mutual information. ArXiv, 2024a.
Xinhao Zhong, Hao Fang, Bin Chen, Xulin Gu, Tao Dai, Meikang Qiu, and Shutao Xia. Hierarchical features matter: A deep exploration of gan priors for improved dataset distillation. ArXiv, 2024b.
13

---

Published as a conference paper at ICLR 2025
A PROOF OF THEOREM
I(X;
P(a, (lv)
PP() P(y)
ΣV
Σ
P(a.9ly) log
by definitions
P(zly))((lv))
WEY
SEY
ΣEΣ
P(x, U. v) log
P(ily)
REX JEY WEY
P()(z))
ΣE
P(x, U. v) log
by Markov chain Y - -
P(Oly)
REX JEY WEY
()(,,))
((...))
ΣEΣ
P(z, 0.) log
P(xx)
P(u)
REX JEZ YE?
P(x, 9) (()))((i)
P(..u) v)
ΣΣΣ
P(x, 9. v) log
P(O)P(y))
xEX JEY NEY
P(r.ū) P()))P(i)
P(ù.v)
Σ Σ REX DEY
P(z, 1) log
ΣΣ
P(ú, v) log
P())P(wv)
DEY WEY
I(X;8) - I(Y;Y),
by definition iI
B WATER-FILLING TO SOLVE (6)
For brevity, let q = q", f = f(x). and == E. H. Then 6
restated as
min (L(pllq),
s.t. |p f-1ll S. AEA
Let p' be the optimal solution. Note that KL (4) is a convex function with the minimum point p p= q. We have p" q if lq fll S e, and 1p" flli E otherwise.
Consider | fl1 > in the following. Since p", A we have
Σ
Ipt fil= Σ
Ipi ful
(8)
ieV: <i<h
Lemma 1. Vi E Y. either yi 2 P 2 Jooqq S p S St.
Proof. Assume that Di E Y. p > q: 2 Si. Since p", q E A there must 3j E Y, p/ < <gj. Let
p - + (p) + log
P +5
s(t) = (p: - { log
qi
qi
Note that gL(E) is continuous and g'(0) log p log rl 4; g(E) < (0). This implies that decreasing p. by g and increasing p by & will reduce KL (1)). contradicting the optimality of p". No constraints are violated if is chosen such that p. and p3 + 1.
0, so 3 0, AA E (0,
q:
f
Let A = {i Y sub-problems, 9 and TO
fi} and B == Y \ A. Based on and Lemma we divide
into two
min Σ
ps log Pi
min
Σ Ps
log
Pi
di
iEA
VEB
s.t.
> Pi
fr =
(9)
s.t.
Σ Pi fr
(10)
2'
2'
iEA
iEB
2 pi 2 f.
i A.
S P;
< fi
i B.
14

---

Published as a conference paper at ICLR 2025
To solve 99, introduce Lagrange multipliers o-
R for Σ SEA
Pj fs
2. and
X 2 for
p fi, E A. The KKT conditions are
Pi - 1¹' - X; 0,
+ log
96
(pî - f1)15
Eliminating X; 2 0 yields
Pi v², 9i
+ log
(11)
Pi 42 0.
(pt fi) 1 log
(12)
qb
If 1 + log fs 2 v*, then pt fi implies 1 + log %
Pi %
> v- contradicting 12 pł fi.
If 1 + log
f %
< v., then 11 implies p. > fú, and (2
implies p. exp((" - -1).
In summary,
1 log f 2 0²
fer
(13)
l expp(e" -1),
other.
Let w exp((* 1) and [3 becomes
p max((fi w (a)), E A,
(14)
where w satisfies
- Let ((w = E max(0, (, wa, ft). which is piecewise-lineat
EEΛ
EEA
increasing function with breakpoints can find w (0, such that c(w3) 5. The constraints q; 2 pf. A are naturally satisfied by w < I.
4s i E A. Note that c(0) 0 and c(1) بلليهماا و.
qi
Solving TT similarly yields
p min(ff w7.), E B,
(15)
where wi satisfies
Σ B
p Ss = - and wh E (1, +00).
Combining 14 15 and w < 1 < wh the solution of is
p min (fi, w ()), wbg), i Y.
(16)
We propose Algorithm to solve ? efficiently. Firstly we address the trivial case. For non-trivial cases, we ensure successively. Once c(we
f1 fz
fizl by sorting, To find w'a, we check पाوl
fı fa 21
91
92
w la we get w + ft [ (1). This process is Σ 9:
4--1
known as water-filling, because w is like a rising water level,
91 92 .... are like ascending steps, Jv. f
fi fa
and 5 is like the total volume of water. To find wū, check symmetric process is known as reverse water-filling. Finally we undo the sorting and return (66. Our time complexity is O( log YY) due to the sorting.
q qii similarly. This
Furthermore, we propose Algorithm 3 to utilize GPU acceleration. Leveraging the PyTorch op- erators, manage to eliminate the if and while statements in Algorithm 2 making Algorithm sequential completely and suitable for GPUs. Algorithm 3 can solve many (f,, q) parallelly one batch, achieving lower computational costs.
15

---

Published as a conference paper at ICLR 2025
Algorithm 2: CPU-based water-filling.
Algorithm 3: GPU-based water-filling.
Input: ft. 9, for E Y. Output: pi for E Y. if Eiey 9 fil S & then
Input: PyTorch tensors f. q of size [Y] Output: PyTorch tensor p of size [
m (ll flli I e);
return qi for E Y;
Reindex fa, qi so that fi
fM 4/v1
Reindex f. q by torch.sort( 4);
j 1;
F +--0x 0:
< fcumsumO;
Q O:
Q < g.cumsum);
while L Q
F < 2
do
M < (1Q F
FS 5)
9;
F F+ F + fj:
Q - Q + ワ3
J } 1;
j - M argminO;
w7 F+1.
WA K- F Fl +
Q
j j |/;
F 0;
- 1 F;
Q - 0;
Q - 1 Q;
while
f3 Q F do
M 4
[QQ F -2);
F 93
t F fz; Q Q ワド
jt 1; 5-
3 <- M. Margmaxx
Fl
w
W
QUi Restore the indices of f. q
Restore the indices of fu, 96: return min(max(fr w q.), w))) for i e Y;
return q. wheree(m, f :clip( q4 w wb)));
C COMPUTATIONAL COST
We evaluate the time cost of Algorithm [ whose convex optimization problem is solved by Al- gorithm We take batch with 512 images and infer 100 times. The time cost measured by an official tool provided by PyTorch. We exclude the time for I/O (i.e. from disk to memory, and from CPU to GPU), only counting the time for computation. Our experiment is conducted on NVIDIA GeForce RTX 3090.
Table 4: Time cost of Algorithm equipped with Algorithm 3
IR-152 & CelebA
IR-152 & FaceScrub
VGG-16 FaceScrub
None SSD
18.63
17.70 s
5.65 s
19,22
18.16 16 s
6.07 s
Increment
3.1%
2.5%
7.4%
Table shows that increase little time. The higher percentage VGG-16 is due to the shallower model architecture. absolute terms, modifying 512 outputs for 100 times only needs 0.5 seconds. If we take the I/O time into account, the percentages will be low enough be ignored.
16

---

Published as a conference paper at ICLR 2025
D HYPER-PARAMETERS OF DEFENSES
Table 5:
of the defenses in our experiments.
IR-152 CelebA
IR-152 & FaceScrub
VGG-16 FaceScrub
MID
B = 0.005
B 0.02
3 0.015
BIDO
A 0.0005, Ay 0.005
A. 0.005, A, 0.05
A 0.0005, A, 0.005
LS
a -0.1
a -005
= -0.05
TL
freeze 50% layers T = 0.1, = 0.35
freeze 50% layers
freeze 50% layers
SSD
T 0.1, = 0.5
T = 0.1, 0.8
E
UNDER RLBMI ATTACK
Table 6: MIA resistance of various defenses under RLBMI attack.
+ Accl
4 Accō
t Seval
1 face
None
44%
65%
505
0.89
MID
31%
53%
523
0.90
BiDO
24%
44%
552
1.01
LS
24%
52%
527
0.98
TL
37%
55%
523
0.94
SSD
21%
42%
572
1.07
We conduct the experiment under RLBMI (Han et al.) 2023), ), a soft-label attack. Aligned with Tables 3 the target models IR-152 trained on CelebA. Due the huge computational cost of RLBMI, we only reconstruct image per label. Table 6 shows that our SSD still outperforms the other defenses.
F COMPARISON WITH PURIFIER
Table 7: Comparisons between Purifier and SSD.
+ Accl
4 Acc5
1 8eval
1 8 face
Purifier
3.6%
8.2%
721
1.43
Mirror
SSD
3.2%
8.2%
728
1.46
Purifier
0.6%
2.6%
723
1.56
C2FMI
SSD
1.2%
1.8%
744
1.59
Purifier 22.8%
41.6%
591
1.14
BREP
SSD
1.8%
4.2%
769
1.62
Purifier SSD
40.2%
57.6%
546
0.99
LOKT
1.2%
9.0%
769
1.61
Acc
Dist
Purifier SSD
84.5%
0.362
Utility
87.1%
0.191
Purifier (Yang et al. 2023) is a black-box defense against membership inference attacks, and perhaps resists model inversion attacks. Despite the lack of details about À and &NN, reproduce their
17

---

Published as a conference paper at ICLR 2025
work setting == k 1. If the L2 distance between the input and the nearest training sample is less than 0.00005, then we swap the top-1 and top-2 labels. The validation set is used to train the CVAE. Aligned with Tables [ the target model is the same IR-152 trained on CelebA. Table shows that our SSD outperforms Purifier.
G HIGH RESOLUTION
We use HDD-CelehA-Croppeflee to generate high resolution images, which are cropped and resized to 224 x 224. The IR-152 and Max models are retrained this new CelebA, and the test accuracy of Max achieves 97.2%.
We select Mírror the attacker, equipped with the 1024 1024 StyleGAN2 trained FFHQQ The generated images are center-cropped to 800 x 800 and resized to 224 x 224. Due to the huge computational cost for high resolution, we only attack the first 20 labels and reconstruct images per label. Table shows that our SSD still outperforms the other defenses.
Table 8: Comprehensive results high resolution.
Mirror
Utility
Kyyee-parameete
Acc1
1 Acc5
1 deval
t 8 face
Acc
Dist
None
32%
53%
492
1.25
96.4%
MID
37%
68%
477
1.28
93.1%
0.334
0.005 X 0.05, X, = 0.5
BiDO
37%
55%
487
1.23
94.7%
0.112
LS
28%
43%
508
1.29
94.1%
0.118
a = -0.005 freeze 80% layers
TL 64%
96%
472
1.17
94.1%
0.719
SSD
17%
33%
522
1.30
95.0%
0.079
T= 0.1, = 1.5
H DISCUSSION ON ADAPTIVE ATTACKS
potential adaptive attack strategy is:
1. Query the same x repeatedly and count the frequency of different outputs.
2. Estimate our sampling probability by the frequency.
3. Infer our original output by the sampling probability.
an online server detects such pattern queries, can block them. Step back and consider again, propose memory- free and low-cost strategy block such adaptive attack:
Design a hash function h: X + N where N is the integer set. When users/attackers query æ. we take h(x)) as the random seed for sampling, ensuring aome-inputsaame-ouppee Locality Sensitive Hash Gionis et al. 1999) can be used to cope with the perturbations to x.
18


Reviews:
[{"review_id": "pDlVkw67yA", "reviewer": "Reviewer_VqZg", "summary": "This paper proposes a post-process black-box MI defense Stealthy Shield Defense (SSD) based on conditional mutual information (CMI). By leveraging CMI, SSD aims to reduce the model's output dependence on its input. The experiments demonstrate SSD's effectiveness against four state-of-the-art black-box MI attacks on two datasets.", "questions": "* [r1] was omitted in the paper. I suggest the author should add [r1] in the experiments\n\n* Can the author evaluate the proposed method's performance on high-resolution images? Additionally, please provide more details on how high-resolution MI attacks like MIRROR are adapted to low-resolution scenarios.\n\n* Algorithm 1 references $D_{valid}$ without a clear definition. Please provide an explicit expression for $D_{valid}$. Furthermore, it would be interesting to understand the impact of $D_{valid}$’s size on the final output. Were all training images used as $D_{valid}$?\n\n* For the LOKT experiments, please clarify whether the authors retrained their TACGAN and other surrogate models or used pre-trained models. Note that LOKT requires training target models with GANs and surrogate models,  which in this case SSD needs to be performed on every samples.\n\n*  Please specify the number of identities used for attacks and the quantity of reconstructed images generated per identity.\n\n* I would like to see a comparison of the prediction time between SSD and baseline methods like NoDef or other defenses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* This work focuses on defending against black-box MI attacks which have not been addressed in the existing defense.\n\n* As a post-processing defense, SSD can be easily integrated with most pre-trained models for defending against black-box attacks.\n\n* Empirical results demonstrate SSD's success in reducing the attack performance of four state-of-the-art black-box MI attacks.", "weaknesses": "* Practical Limitations: Although the idea of post-processing defense is interesting, the proposed method raises concerns about its applicability in real-world scenarios. To modify the model's prediction output, **SSD requires a dataset $D_{valid}$**, which I believe should be real data (either the training dataset or its validation set). This means the user must store raw training data or predictions on the training data to perform predictions, potentially increasing the risk of data leakage.\n\n* SSD's prediction process involves an optimization step for each image, leading to significantly increased computational costs and slower inference times compared to other models.\n\n* The experiments were conducted on low-resolution 64x64 images, limiting the generalizability of the findings high-resolution scenarios.\n\n* [r1] was omitted in the paper while it is also a state-of-the-art black-box attack.\n\n* The paper suffers from some typos and grammatical errors. For examples, lines 41, 107.\n\n[r1] Han, Gyojin, et al. \"Reinforcement learning-based black-box model inversion attacks.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fLHm6p4ATL", "reviewer": "Reviewer_Ui99", "summary": "This paper proposes a post-processing defense SSD based on conditional mutual information (CMI) especially for defending against black-box model inversion attacks. It theoretically proves that CMI is effective and further reduce modifications to outputs by proposing adaptive rate-distortion framework. Experiment results indicate that SSD achieves SOTA better utility-privacy trade-off.", "questions": "-\tHow does SSD perform when the distribution of the public dataset differs significantly from the private dataset (e.g., public dataset = FFHQ, private dataset = CelebA)? Clarification on this point would help to understand SSD’s adaptability under various deployment conditions.\n-\tThe paper states that SSD shows exceptional performance in reducing attack accuracy for LOKT and BREP methods. Could the authors elaborate on why SSD is particularly effective against these specific methods?\n-\tCould the authors provide additional details on the GAN prior training and any specific hyperparameters used for Figure 1?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-\tThis paper considers defense against model inversion attack from a new perspective by proposing CMI, which provides new insights into this field.\n-\tIt provides thorough theoretical analysis to support the effectiveness of the SSD defense, with a clear and rigorous logical structure.\n-\tThe paper achieves state-of-the-art performance in terms of utility-privacy trade-off and defense against advanced black-box attacks, qualitatively demonstrating its effectiveness.", "weaknesses": "-\tKey concepts, such as the water-filling method, lack adequate coverage in the main body, making it challenging for readers to fully grasp the methodology without delving into appendices. Greater emphasis on these aspects within the main text would improve the paper's accessibility.\n-\tThe experimental setup omits certain critical information, such as the dataset used for GAN prior training and specifics for Figure 1, leaving questions regarding reproducibility and completeness.\n-\tThe study exclusively benchmarks against state-of-the-art white-box defenses but omits comparisons with black-box defenses, which would be more pertinent for the black-box MIA context.\n\n\nMinor remarks:\n\n-\tThere are also minor errors, such as duplicated words (line 107), inconsistent notation (duplicate use of $p$ at line 215) and typo at line 314.\n-\tThe notation is unconventional, such as using non-bold font for vector inputs like $\\mathbf{x}$, which reduces consistency with standard notation practices.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2PzuicgikW", "reviewer": "Reviewer_KsSS", "summary": "This paper proposes a post-processing based defense to protect against black-box model inversion attack. The key insight of the paper is to transform the output of the model to reduce conditional mutual information (CMI). This is done to reduce the dependence between the output and the input while preserving the dependence on the true labels. The authors develop a mechanism to transform the model’s prediction to reduce CMI by framing it as an optimization problem. The experimental results on multiple image classification datasets show that their proposed SSD defense provides a better privacy-utility tradeoff compared to prior defenses.", "questions": "1. In line 228, the authors state that “the objective function is too complex for the convex optimizer to solve.” and use this as the motivation to minimize a simplified objective $KL(p||q^y)$ instead (by sampling $y\\in\\mathbb{Y}$). Why is this the original objective too complex? Wouldn’t you be able to use gradient descent to solve for $p$? Would this lead to a better solution?\n2. How was the temperature $T$ picked in Algorithm 1?\n3. What was the value of $\\epsilon$ for the proposed defense in the experiments?\n4. Why is the Acc@1 for IR-152 lower for “no defense” compared to prior defenses? This suggests that adding prior defense improves the attack success rate, which is very strange.\n5. Can you address the adaptive attack discussed in Weakness#1?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper is well-written and easy to follow.\n2. The idea behind the proposed defense is intuitive and presented well.\n3. Experiments seem comprehensive and the defense shows better utility-privacy tradeoff compared to prior defenses in both soft and hard label settings\n4. The defense is post-processing based, making it easy to adopt.", "weaknesses": "1. The proposed defense could be susceptible to an adaptive attack. An adversary could query the same input multiple times to obtain multiple predictions from the model. Since the defense produces outputs by perturbing the original prediction, the adversary could compute an average over multiple outputs to get a better estimate of the model’s true output. Such an adaptive attack is not discussed by the paper.\n2. The defense requires a validation dataset to implement, which could limit its adoption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7iyU0MX9LH", "reviewer": "Reviewer_J1L4", "summary": "This paper addresses the critical issue of protecting machine learning models from black-box model inversion attacks (MIAs). Specifically, the attacker’s objective is to reconstruct private training data by only accessing the model's outputs.The authors therefore propose a novel defense mechanism named Stealthy Shield Defense (SSD) to post-process the model’s output such that the information revealed by the model’s output about the private data is minimized under constrain. This method leverages Conditional Mutual Information (CMI) to reduce the dependency between the model's output and private data. In addition, they also propose an adaptive rate-distortion framework using water-filling method to preserve the utility while minimizing the CMI.", "questions": "1. The proposed scheme requires iterating over all possible labels to perform the defense. What happens if there are a large number of possible labels? For example, a large company or region may train a face recognition model that includes thousands or millions of faces. Will this scale slow down the defense and become a bottleneck? I suggest some detailed computational analysis.\n\n2. As the authors pointed out in the paper, X→Z→Y ̂ is a Markov chain. It is possible that when input data has large dimensionality, or the model has very deep layers. Under this circumstance, it is likely that Y ̂ shares very little information with X. The derived MI I(X;Y ̂ ∣ Y) will have such a small value that could be hardly optimized. Does the proposed scheme still hold effective in this scenario? The authors should provide more empirical/theoretical analysis to justify this. \n\n3. Are there any manifest adaptive attack schemes that can target at this defense? How difficult is it to design/launch such attacks?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The authors propose SSD, a post-processing defense mechanism that doesn’t require retraining the model, making it practical for real  deployment. \n\n2. The authors theoretically prove that minimizing CMI serves as a special information bottleneck, therefore minimizing CMI can effectively balance data privacy and utility. By iterating the CMI through all possible labels, the whole dataset can thus be. \n\n3. The paper introduces an adaptive rate-distortion mechanism optimized using the water-filling method. This approach efficiently calibrates the probability distributions output by the model. \n\n4. The authors validate their method across various attack algorithms (BREP, Mirror, C2F, LOKT), datasets (FaceScrub, CelebA), and model architectures (VGG-16, IR-152). The results demonstrate that SSD outperforms existing defenses in terms of defense scheme robustness and effectiveness on preservation of model utility.", "weaknesses": "1. Even though the authors claim that the computational overhead is negligible due to the efficient optimization on GPU, a more detailed analysis or benchmarking of the computational cost would greatly support this claim. \n\n2. Using MI/CMI in deep learning is gaining increasing attention. However, its introduction in related work lacks both depth and breadth, which makes it hard to find the role of this work into the relevant community. \n\n3. The authors should use a deeper model architecture and higher-dimensional input data for training and prove the effectiveness of the proposed method. When the input data is in high dimensionality, it usually contains a significant amount of irrelevant information. Even worse, since the model depth is also high, the final output, Y ̂, may only contain a small amount of MI with the input, X. I am wondering if optimizing I(X;Y ̂ ∣ Y) will be challenging in this case. \n\n4. Even though this is a robust scheme against model inversion attack, the authors should discuss about the potential possibility of adaptive attacks. If the adaptive attack is unlikely to happen for now, the authors should also state the reason why.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)