Table 1: According to the attack effect of IN, we adjust the hyperparameters of different defenses to provide the defended model with three levels of privacy protection, resulting in a gradual decrease in model accuracy (original Acc=79.97 %). All experiments are performed on Block3. Implementation details and hyperparameters of defenses are in Appendix B.4.
Defense	Method	Acc = 79.97%	Acc = 79.23%	Acc = 78.56%
LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓
NoPeek	rMLE	0.853	0.006	7.059	1.048	0.855	0.006	7.099	1.039	0.856	0.006	7.105	1.038
LM	0.679	0.233	11.74	0.449	0.729	0.171	11.00	0.500	0.766	0.178	10.29	0.539
IN	0.638	0.376	15.57	0.160	0.607	0.377	15.51	0.156	0.618	0.352	15.15	0.168
GLASS	0.293	0.554	17.53	0.142	0.351	0.422	14.68	0.244	0.414	0.343	13.46	0.326
GLASS++	0.220	0.678	21.60	0.070	0.295	0.549	18.04	0.160	0.301	0.532	18.21	0.115
Defense	Method	Acc = 79.34%	Acc = 79.26%	Acc = 79.04%
LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓
DISCO	rMLE	0.618	0.330	14.69	0.228	0.784	0.147	11.87	0.367	0.763	0.162	11.96	0.361
LM	0.387	0.620	19.96	0.127	0.762	0.220	12.48	0.330	0.799	0.205	11.48	0.422
IN	0.410	0.584	20.59	0.049	0.626	0.336	15.84	0.149	0.636	0.314	15.56	0.158
GLASS	0.165	0.776	24.76	0.022	0.328	0.464	16.37	0.154	0.345	0.431	15.55	0.201
GLASS++	0.152	0.784	24.75	0.021	0.241	0.584	18.79	0.082	0.260	0.576	18.46	0.086
Defense	Method	Acc = 79.13%	Acc = 78.76%	Acc = 78.13%
LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓
Adv-Learning	rMLE	0.591	0.327	14.60	0.252	0.654	0.200	12.69	0.343	0.678	0.155	12.03	0.381
LM	0.308	0.667	20.80	0.139	0.414	0.533	17.19	0.217	0.480	0.427	15.04	0.258
IN	0.483	0.517	19.82	0.059	0.584	0.382	17.57	0.100	0.601	0.342	16.45	0.130
GLASS	0.155	0.765	23.68	0.067	0.208	0.626	18.26	0.148	0.197	0.686	20.17	0.110
GLASS++	0.131	0.816	25.93	0.016	0.142	0.794	24.69	0.023	0.157	0.763	23.21	0.040
Defense	Method	Acc = 78.57%	Acc = 78.04%	Acc = 77.83%
LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓
Noise Mask	rMLE	0.703	0.227	14.65	0.188	0.771	0.099	11.58	0.378	0.835	0.021	7.90	0.869
LM	0.612	0.409	18.30	0.079	0.708	0.225	14.61	0.185	0.791	0.057	8.78	0.709
IN	0.352	0.610	21.20	0.042	0.424	0.549	19.71	0.059	0.590	0.424	16.22	0.133
GLASS	0.230	0.660	21.66	0.039	0.245	0.607	20.16	0.056	0.360	0.396	15.18	0.174
GLASS++	0.184	0.714	22.57	0.032	0.231	0.626	20.52	0.051	0.283	0.524	17.71	0.096
Defense	Method	Acc = 78.09%	Acc = 77.65%	Acc = 77.35%
LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓
Dropout Defense	rMLE	0.455	0.563	20.67	0.048	0.539	0.522	19.58	0.062	0.610	0.426	17.29	0.110
LM	0.319	0.717	23.44	0.026	0.408	0.659	22.21	0.034	0.483	0.606	21.11	0.044
IN	0.350	0.612	21.12	0.043	0.425	0.555	20.21	0.053	0.499	0.490	18.83	0.073
GLASS	0.144	0.790	25.31	0.018	0.171	0.766	24.23	0.023	0.195	0.712	22.83	0.031
GLASS++	0.144	0.797	25.30	0.018	0.163	0.777	24.46	0.022	0.169	0.766	24.23	0.022
Defense	Method	Acc = 79.23%	Acc = 78.65%	Acc = 77.73%
LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓
Shredder	rMLE	0.680	0.231	14.38	0.217	0.700	0.200	13.67	0.250	0.676	0.234	14.37	0.220
LM	0.547	0.475	19.09	0.074	0.564	0.448	18.44	0.086	0.538	0.481	18.97	0.072
IN	0.373	0.604	21.03	0.044	0.383	0.602	20.94	0.044	0.390	0.596	20.87	0.045
GLASS	0.222	0.668	21.82	0.042	0.246	0.634	20.95	0.047	0.236	0.646	21.32	0.045
GLASS++	0.207	0.695	22.27	0.034	0.213	0.695	22.19	0.035	0.211	0.697	22.06	0.036
Defense	Method	Acc = 78.74%	Acc = 78.44%	Acc = 77.56%
LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓	LPIPS ↓	SSIM ↑	PSNR ↑	MSE ↓
Siamese Defense	rMLE	0.745	0.155	12.78	0.304	0.740	0.176	12.74	0.308	0.746	0.175	12.48	0.324
LM	0.574	0.451	16.97	0.157	0.604	0.405	16.40	0.176	0.670	0.335	14.67	0.228
IN	0.378	0.614	21.29	0.041	0.386	0.592	21.00	0.044	0.425	0.553	20.28	0.052
GLASS	0.166	0.797	25.30	0.023	0.186	0.751	23.77	0.033	0.201	0.731	23.07	0.046
GLASS++	0.144	0.826	26.30	0.016	0.159	0.802	25.33	0.022	0.159	0.797	25.14	0.021