Privacy-Preserving Vision Transformer on Permutation-Encrypted Images
Authors: Fusheng Hao
Fengxiang He
Yikai Wang
Fuxiang Wu
Jun Cheng
Dacheng Tao
Venue: Submitted to ICLR 2023 (2023)
Keywords: vision transformer
privacy

Abstract: Massive human-related data is collected to train neural networks for computer vision tasks. Potential incidents, such as data leakages, expose significant privacy risks to applications. In this paper, we propose an efficient privacy-preserving learning paradigm, where images are first encrypted via one of the two encryption strategies: (1) random shuffling to a set of equally-sized patches and (2) mixing-up sub-patches of the images. Then, a permutation-equivariant vision transformer is designed to learn on the encrypted images for vision tasks, including image classification and object detection. Extensive experiments on ImageNet and COCO show that the proposed paradigm achieves comparable accuracy with the competitive methods. Moreover, decrypting the encrypted images is solving an NP-hard jigsaw puzzle or an ill-posed inverse problem, which is empirically shown intractable to be recovered by the powerful vision transformer-based attackers. We thus show that the proposed paradigm can destroy human-recognizable contents while preserving machine-learnable information. Code will be released publicly.

Under review as a conference paper at ICLR 2023
PRIVACY-PRESERVING VISION TRANSFORMER ON PERMUTATION- IMAGES
Anonymous authors Paper under double-blind review
ABSTRACT
Massive human-related data is collected to train neural networks for computer vi- sion tasks. Potential incidents, such as data leakages, expose significant privacy risks to applications. In this paper, we propose an efficient privacy-preserviee preserving learning paradigm, where images are first encrypted via one of the two encryption strategies: (1) random shuffling to a set of equally-sized patches and (2) mixing- up sub-patches. Then, a commutatioo-qquivrriin learn on the encrypted images for vision tasks, including image classification and object detection. Extensive experiments on ImageNet and COCO show that the proposed paradigm achieves comparable accuracy with the competitive methods. Moreover, decrypting the encrypted images is solving an NP-hard jigsaw puzzle an ill-posed inverse problem, which is empirically shown intractable to be re- covered by the powerful vision transformer-based attackers. We thus show that the proposed paradigm can destroy human- recoonnzaee contents while preserv- information. Code will be released publicly.
vision transformer is designed to
ing
INTRODUCTION
Deep models trained on massive human-related data have been dominating many computer vision tasks, e.g., image classification He al. (2016), face recognition Li al. (2021), etc. However, most existing approaches are built upon images that can be recognized by human eyes, leading to the risk of privacy leaks since visually perceptible images containing faces or places may reveal privacy-sensitive information. This could raise concerns about privacy breaches, thus limiting the deployment of deep models in privacy-sensitiveleee wes-essiivveeeeurit--rtttee criiticaa application scenarios or increasing people's doubts about using deep models deployed in cloud environments.
To address the increasing privacy concerns, researchers have integrated privacy protection strategies into all phases of machine leaminng, such data preparation, model training and evaluation, model deployment, and model inference Xu et al. (202T). The emerging federated learning allows multi participants jointly train machine learning model while preserving their private data from being exposed Liu et al. (2022). However, attackers can still recover images with high accuracy from the leaked gradients Hatamizadeh al. (2022) confidence information Fredrikson et al.) (2015). To data in a confidential level, directly learning and inferencing on encrypted data is emerging as promising direction. Unfortunately, two huge complications remain Karthik ett al. (2019): (1) the encryption methods themselves, such as fully homomorphic encryption, have very high computation complexity and (2) training deep models in the encrypted domain is an extremely challenging task due to the need for calculations in the ciphertext space.
protect privacy-sensitee
Recent studies in the natural language processing field suggest that higher-order co-occurtence statistics of words play a major role in masked language models like BERT (iinaa et al. (20TT). Moreover, has been shown that the word order contains surprisingly littl information compared to that contained in the bag of words, since the understanding of syntax and the compressed world knowledge held by large models (e.g. BERT and GPT-2) are capable to infer the word order Malkin et al. (2021). Due to the property of attention operation, when removing positional encoding, Vision Transformer (ViT) Dosovitskiy et al. (2020) is connutation-equivaatnn As evaluated by experiments, removing the positional embedding from ViT only leads to moderate performance drop (%3.1, please see Table Tb. Such phenomenon inspires us to explore permutation-baseee encryption strategies.
w.r.t. its attentive tokens.

---

Under review as a conference paper at ICLR 2023
Original imaiges
Irages emerypted RSS
Images everypted by MI
Images munryptd
Figure 1: Illustration of images encrypted by random shuflling (RS), mixing-up (MI), and their com- bination. The visual contents of encrypted images are near-completely protected from recognizing by human eyes.
To maximize the usability of the resulting paradigm, the following two requirements need to be sat- isfied. Firstly, the encryption process should preserve the machine-learnable information of inputs. Compared with existing deep models applied on the non-encrypted images, the performance drop expected to be insignificant and acceptable, and thus make it possible to replace existing deep mod- circumstances. Secondly, the human- recoonizaee contents
els on large scale for privacy-sensiigg should be largely interfered, and the decryption algorithm is expected to have a very high complexity or an unaffordable cost. In addition, it would be better if the encryption algorithm decoupled from the models to be trained, as this would allow the encryption of image contents in scenarios where only limited computing resources are available.
To this end, propose an efficient privacy-preserving learning paradigm. The key insight of our paradigm is two-fold: (1) designing encryption strategies based on sermutation-equivariann (2) making part of the whole network cornuutttioneqqivariit
and which allows it to learn the encrypted images. Two strategies are proposed to encrypt images: random shuffling (RS) images to set of equally-sized patches and mixing-up (MI) sub-patches; Figure Decrypting an image encrypted by RS is solving a jigsaw puzzle problem, which can incur a large computational overhead since the problem to be solved is an NP-hard one Demaine & Demaine (2007). Decrypting an image encrypted by MI is solving an ill-posed inverse problem, which could be hard to solve due to the difficulty in modelling the sub-patch distribution. We indicate that both kinds of encrypted images are still machine-learnable by further designing architectures PEVIT and PEYOLOS, based on ViT and YOLOS Fang et al. (202T) for the image classification and object detection tasks, respectively: see Figure? 2 Specifically, our main contributions are summarized as follows:
We propose an efficient privacy-preserving learning paradigm that can destroy human- recognizable contents while preserving machine-learnable information. The paradigm adopts decoupled encryption process that utilizes the permutation-equiiarianne property so as to be still learnable for networks that are (partially) permutation--qquivriia
RS is tailored for the standard image classification with vision transformers. By substituting reference-basee positional encoding for the original one, the network is capable of learning images encrypted by RS.
Another hallmark of our paradigm is that by further designing MI, the paradigm is extensible position-sensitive tasks, such as object detection, for which we adapt the way that image patches are mapped make the network partially Extensive attack experiments show the security of our encryption strategies. Comparison results on large-scall scale benchmarks show that both PEVIT and PEYÓLOS achieve promising performance even with highly encrypted images as input. We thus show that the proposed paradigm can destroy human-rees remaan-ecognizabbe contents while preserving machine-learnable information.
cemmutation-equvariinn
2

---

Under review as a conference paper at ICLR 2023
mink
mink badger paic..
bodger
VIT
PEVIT
post...
exuguu
cal
class label
image creyytas
class label
YOLOS
tai
PEY
bounding box
imggeroyyddd MI
bounding box
Baseline
Ours
Figure 2: Images enerypted by RS and MI are still
by further designing archi-
tectures PEVIT and PEYOLOS, based on ViT and YOLOS.
RELATED WORK
Vision transformers. Self-attention based Transformer Vaswani al. (2017) has achieved great success in natural language processing. To make Transformer suitable for image classification, the pioneering work of ViT Dosovitskiy al. (2020) directly tokenizes and flattens 2D images into sequence tokens. Since then, researchers have been working on improving Vision Transformers and examples include DeiT Touvron al. (2020), TiT-ViT Yuan et al. (2021). PVT Wang et al. (2021), and Swin-Transfone Transformer Liu et al. (2021). In addition, the intriguing properties of ViT are investigated in Naseer et al. (2021).
Jigsaw puzzle solver. The goal of jigsaw puzzle solver is to reconstruct an original image from its shuffled patches. Since this problem NP-hard Demaine & Demaine (2007), solving jigsaw puzzles of non-trivial size is impossible. Most of the existing works in computer vision focus on the jigsaw puzzle problem composed of equally-sized image patches and examples include the greedy algorithm proposed in [Coo et al. (20T0), the particle filter-based algorithm proposed in Yang et al.) (21II) the fully-automatic solver proposed in Pomeranz et al. (201T), and the genetic-basee solver proposed in Sholomon al. (2013).
machine learning. The aim privacy-preservie machine learning inte- grate srivacc-preserving techniques into the machine learning pipeline. According the phases of privacy integration, existing methods can be basically divided into four categories: data preparation, model training and evaluation, model deployment, and model inference Xu al. (202T) Federated learning allows multiple participants to jointly train machine learning model while preserving their private data from being exposed Liu al. (2022). However, the leakage of gradients Hatamizadeh et al. (222)) or confidence information Fredrikson et al. (2015) can be utilized to recover original images such as human faces. From the perspective of data, enerypting data and then learning and in- ferencing on encrypted data can provide strong privacy guarantee, called confidential-leww privacy, which receives increasing attention recently. Both homomorphie encryption and functional encryp- tion have been employed to encrypt data due to their nature of allowing computation over encrypted data, which enables machine learning on encrypted data Xu et al. (2019); Karthik et al.) (2019). However, scaling them to deep networks and large datasets still faces extreme difficulties due to the high computational complexity involved. The block- wise pixel shuffling encryption strategy Tanaka (0188; ; Madono et al. (2020) and the pixel-based image eneryption strategy Sinchotedunnrong al. 2019 have low computational complexity, but scaling them to position-sensitie tasks faces great challenges. In contrast, our permutation-bast based encryption strategies simple, efficient and easy to implement, and can be applied to position-sensitio tasks.
METHOD
In this section, we propose an efficient privacy- presevvni learning paradigm that can destroy hucon-bcco contents while preserving machine-leamable information. We first provide the permutation- based encryption strategies, in Sec. Sec.B.]] Followed by the designed VIT adaptions which (partially) permutation-equiivrinnt
handle the encrypted inputs, in Sec. 3.3 and Sec. 3.4
3

---

Under review as a conference paper at ICLR 2023
3.1 DESTROYING
CONTENTS
We first consider typical vision tasks which are not quite position-sensitive,, such as image classifica- tion that predicts the global category. Here, Random Shuffling (RS) images to a set of equally-sized image patches can destroy human-recognizalle ple, easy to implement, and decoupled from the network optimization. Under the cireumstance, decrypting an image is solving jigsaw puzzle problem, which can incur large computational overhead since the problem to be solved is an NP-hard one Demaine Demaine (2007). In par- ticular, the dimension of the key space when applying the shufile-nncrypted strategy is the number of puzzle permutations. For an image with N patches, the dimension of the key space is given by Kg N!. For example, 7 7 puzzle has 49! x 6 x 10 possible permutations. Based this, it is easy to further increase the complexity of decrypting an image, by reducing the patch size of puzzles increasing the resolution of the image. Besides, the complexity of decrypting an image can be further increased by dropping some image patches, as experimentally evaluated in Figure 5 and Figure 4
contents. This shuffle-encrypted process is sim-
However, always shuffling image patches is prone to underestimating the position information. To further make our paradigm applicable to position-sensitive tasks like object detection, where precise positions of bounding boxes are predicted, we design another encryption strategy named Mixing (MI). Specifically, MI mixes sub-patches of image patches to destroy human- recognizabl contents while preserving the position information, so that the encrypted data can be learned by networks that are partially
Assuming that there are M sub-patches extracted from an image patch x², the mixing-encrypted strategy can be formulated as follows,
M
where s. denotes the i-th sub-patch of x!, Since the sum function is permutation-cquivariant. MI makes an encrypted image partially permuaaion-cquivaaiaat to its sub-patches. With the MI encryption process, decrypting patch is solving the following ill-posed inverse problem,
M Σ 12
arg min
Both modeling the sub-patch distribution and restoring the sub-put sub-patch order make decrypting an patch a great challenge. Decrypting an image of N patches magnifies this challenge by a factor of N.
3.2 BUILDING BLOCKS OF VIT
In this part, we analyze how the change of input permutation affects each component of ViT.
Self-attention. The attention mechanism is a function that outputs the weighted sum of set of k value vectors (packed into V E R4xd). The k weights are obtained by calculating the similarity between a query vector g E Rd and set of key vectors (packed into K E RR using inner products, which are then scaled and normalized with a softmax function. For a sequence of N query vectors (packed into Q E IR N xd), the output matrix O (of size x d) can be computed by,
0 Attention (Q, K.V) V) Softmax(QK") /VTT/V
(3)
where the Softmax function is applied the input matrix by rows
In self-attention, query, key, and value matrices are computed from the same sequence of input vectors (packed into X IR. Nxd) using linear transformations: Q XW K XWK V XWVV Since the order of Q, K, and V is co-variant with that of X, the permutation of the input of self-attention permutes the output.
Multi-head self-attention. Multi-head Self-Attention (MSA) Vaswani et al. (2017) consists of self-attention layers, each of which outputs a matrix of size N d. These h output matrices are then rearranged into N x dh matrix that is reprojected by a linear layer into N x d,
MSA- Concat(head), head,,) W.
(4)
4

---

Under review as a conference paper at ICLR 2023
Hoad
PEVIT
RPB
RFB
KPG
KPG
LPP
RFD
RPS
Linear Projection
Figure Architecture overview of PEVIT with RPE.
where head; = Attention(Q2, K,, Vi). Qi = XW K2 = XW Vi XWV, and W e Radxxd The order of each head co-variant with that of X. Since the Concat operation con- catenates the vectors from different heads the same position, it is co-variant to the order of X. Therefore, we conclude that the permutation of the input of MSA permutes the output.
Layer normalization. Layer normalization Ba al. (016) applied the last d dimensions, based on which the mean and standard deviation are calculated. Therefore, the permutation of the input of layer normalization permutes the output.
Residual connection. The residual connection He et al. 2016) can be formulated as F(X) X. If the order of the nonlinear mapping F(-) is co-variant with that of X, the residual connection is co-vart co-variant with that of X.
Feed-forward network. Feed-Forward Network (FFN) consists two linear layers separated by a GELU activation Hendrycks & Gimpel (2016). The first linear layer expands the dimension from D 4D, while the second layer reduces the dimension from 4D back to D. Since FFN is applied on the last d dimensions, the permutation of the input of FFN permutes the output.
3.3 CLASSIFICATION ON ENCRYPTED IMAGES
standard method handle images in ViT, the fixed-size input image of H W x C' is decom- posed into batch of N patches of fixed resolution of P x P. resulting in sequence of flattened 2D patches X Nx CC) For example, the sequence length N HW/P" P2 of VIT could be
196 for image classification on the ImageNet dataset. To destroy the human-recognizzbble contents, we choose RS the encryption strategy to encrypt images. The reason two-fold: (1) The key space of an image encrypted by RS is big enough and (2) The drop in performance is insignificant. To learn on the images encrypted by RS, we design Permutation- Eqquivaainnn ViT (PEViT), defined as follows,
{x X
Shuflling
{x
x N:
(5)
Zg = [Xelasi x
E:
x
E:
E].
E E ((P
(6)
z'z MSA(LN
Zz-1,
= I...L
(7)
2, FFN(LNN(2 e)) + z'e.
1.....
y L.()))).
where E denotes the linear projection that maps each vectorized image patch to the model dimension D, and Xelass denotes the class token (z) Xelass), whose state the output of the visual transformer (z%) serves as the image representation y.
The differences between PEVIT and vanilla VIT are two-fold: (1) Our model takes shuffled patch embeddings as input and (2) The learned positional encodings are removed. Since the permutation of the input of all the building blocks permutes the output, the order of z is co-variant with that of zn- It is worth noting that the class token is fixed in 2g. Therefore, z,, corresponds to the image representation y that is equivariant the order of patch embeddings image patches.
5

---

Under review as a conference paper at ICLR 2023
When directly introducing positional encoding, the cermuutaion-equivariaan property of PEViT is destroyed. Inspired by relative encoding Shaw et al. (2018). we propose a reference- based positional embedding approach that can retain the permutation sermutation-quuivariinne property,
Eos (xf) RPF(x,"
x
(10)
where x E I2 denotes the learnable reference point and RPE denotes the reference-baace posi- tional encoding network that consists of two linear layers separated by GELU activation Hendsy Gimpel (2016), (2016), followed by a sigmoid function. This reference- positional embedding relies only on the learnable reference point and patch embeddings, and thus its order is co-variant with that of input vectors. When added to the patch embedding as follows,
20
Xctass x Epss )); x E Epos (x)): :.:;; x Exos(x2))
(11)
where the cormuuttiionequivrriinn property of PEVIT is retained. An illustration of PEVIT with RPE is depicted in Figure 3
3.4 OBJECT DETECTION ON ENCRYPTED IMAGES
YOLOS Fang et al. 2021 is an object detection model based on the vanilla VIT. The change from a ViT to a YOLOS involves two steps: (1) Dropping the class token and appending 100 randomly initialized learnable detection tokens to the input patch embeddings and (2) Replacing the image classification loss with the bipartite matching loss to perform object detection in a set prediction manner manner(Carion al. (2020).
Since position information plays key role in the low-level object detection task, directly encrypting images with RS messes up the patch positions and thus leads to significant performance degradation. This brings a great challenge to destroy human-recognizable contents while preserving machine- learnable information for the object detection task. We address this challenge by adapting the way image patches are mapped. For an image patch (x.") of a fixed resolution P' x P. we further decom- pose the patch into a batch of sub-patches of a fixed resolution & P Then, these sub-patches are encrypted with MI, resulting in the encrypted patch x-- Accordingly, the input to YOLOO is adapted follows,
2
Zo = | DET.
X100
DET. H((x)): H(xx)) H(x+))+ +P,
(12)
where H denotes nonlinear mapping composed of two linear layers separated by GELU activa- tion Hendrycks & Gimpel (2016) and P E [2(00+N)XDD denotes the learnable positional embed- dings. This adaptation makes YOLOS partially permutation-equivarian to its sub-patches. With our PEYOLOS, we can destroy rroonnrbcogn contents while preserving machine-learnabl information for the object detection task. More details can be found in the appendix.
3.5 DISCUSSION ON THE CONNECTION BETWEEN RS AND MI
RS shuffles the patch order of an image, which destroys the position configurations between patches. MI mixes the sub-patches in a patch, which preserves the position configurations between patches, and can thus be applied to position-sensitive scenarios. Since MI encrypts images only at the sub-patch patch level, it is also applicable to position insensitive scenarios such as image classifica- tion. Considering that key patch of an image encrypted by RS may reveal information identify an object person, RS and MI can even be combined to further enhance the degree of data protection; see Figure
EXPERIMENTS
In this section, first contrast the performance of the proposed PEVIT and PEYOLOS on image classification and object detection tasks, respectively. Due to limited space, ablation studies are provided in the appendix.
4.1 EXPERIMENTAL SETTINGS
Datasets. For the image classification task, we benchmark the proposed PEVIT on ImageNet- 1K Deng al. 2009), which contains w1.28M training images and 50K validation images. For the

---

Under review as a conference paper at ICLR 2023
Table 1: Comparison of different methods ImageNet-1 classification.
image size
ImageNet
method
#param.
FLOPs
top-1 acc.
RegNetY- 16G Radosavovic et al.
(2020)
224
84M
16.0G
82.9
EffNet-B3 Tan & Le
(0119
300
12M
1.8G
81.6
EffNet-B4 Tan Le
2019
380
19M
4.2G
82.9
Ef Tan & Le
2019)
4562
30M
9.9G
83.6
DeiT-B Touvron et al.
2020
224
86M
17.5G
81.8
TNS-B Han al. [222
224
66M
14.1G
82.8
T2T-Vi1-14 Yuan al.
2021
224
22M
5.2G
81.5
T2T-ViT-24 Yuan et al.
202
224
64M
14.1G
82.3
PVT-Large Wang ct al.
2021
2242
61M
9.8G
81.7
Swin-T Liu et
2021
224
28M
4.5G
81.2
Swin-S Liu et al.
(2021
224
50M
8.7G
83.2
GG-T et al.
(2021)
2242
28M
4.5G
82.0
GG-S Yu et
[2021
224
50M
8.7G
83.4
ViT-B/T6 Dosovitskiy et al.
2020)
3842
86M
55.4G
77.9
ViT-L/16 Dosovitskiy et al. 2020
3842
307M
190.7G
76.5
DeîT-B on images encrypted by RS
224
86M
17.5G
35.9
DeïT-B on images encrypted by MI
224
86M
17.5G
78.0
PEViT-B on images encrypted by RS PEViT-B on images encrypted by RS + MI
224
86M
17.5G
78.7
2242
86M
17.5G
69.5
PEVIT-B with RPE on images encrypted by RS
224
87M
17.6G
79.7
object detection task, we benchmark the proposed PEYOLOS on COCO Lin et al. (2014). which contains 118K training. 5K validation and 20K test images.
Implementation details. D We implement the proposed PEVIT based on the Timm library Wight- man (2019). We adopt the default ryper-parameters
the DeiT training scheme Touvron et al. (2020) except setting the batch size to 192 per GPU, where 8 NVIDIA A A100 GPUs are used for training. is worth noting PEVIT (w.o RPE) is equivalent to removing positional embeddings from DeiT. We implement the proposed PEYOLOS based the publicly released code in Fang et al. (202 and change the way image patches are mapped.
Baseline. We propose an efficient privacy-preserving learning paradigm with the aim of destroying human-recognizzbble contents while preserving VIT and PEYOLOS inherited from DeiT' Touvron al. (2020) and YOLOS Fang al. (2021) respectively, which are thus selected as baselines. is worth noting that both PEVIT and PE PEYOLOS designed to be high-performance models that beats state-of-the-art image classification and object detection models, but to unveil that destroying human-recognizablle contents while preserving machine-lcamable information is feasible.
information. The proposed PE-
Mcasurement of privacy protection. As shown in Figure | the visual contents of encrypted images nearly-completely protected from recognizing by human eyes. To measure the strength of privacy protection, we try to restore the original images with various attack algorithms, includ- ing puzzle solver attacker, gradient leakage attacker, and reconstruct attacker. Then, the quality of restored images can reflect the strength of privacy protection. We only provide attack experi- ments from puzzle solver attacker in the main text and more attack experiments can found in the appendix.
4.2 IMAGENET- 1K CLASSIFICATION
The summary of the image classification results is shown in Table It can be observed that (1) Our performance drop only 3.9% compared with the state-of-the-arr CNN-based method Bff (2) Our performance drop is only 2.1% compared with the baseline method DeiT-B, and (3) MI is applicable to image classification. It is worth noting that although the performance of PEVIT-B not state-of-the-art it is applied on encrypted images where the visual contents of images cannot
The pscudocode of RS and MI Py Torch-like style shown in the appendix.

---

Under review as a conference paper at ICLR 2023
images
KENO
Image Top
Imsge siae
Top
Figure 4: Reconstructed images by the jigsaw puzzle solver proposed in Paikin & Tal (015). Here, the effect of image size on image reconstruction quality and classification performance is investi- gated.
Table 2: Object detection performance the COCO test2017 dataset. FPS is measured with batch
size 1 on single 1080Ti GPU. method
backbone
size
AP
params.
FLOPs
FPS
YOLOS-Ti
DeiT-Ti
512 X 28.7
6.5M
18.8G
60
Deformable DETR
FBNet-
800
27.99
12.2M
12.3G
35
YOLOSv4-Tiny
COSA
416
21.7
6.1M
7.0G
371
CenterNet
ResNet- 18
512 * 128.1
129
PEYOLOS on images encrypted by MI
DeiT-Ti
512
25.3
7.1M
19.0G
58
DETR
R8 18
800 X * 336.9
29M
129G
7.4
YOLOS-S
DeïT-S
800
36.1
31M
194G
5.7
PEYOLOS on on images encrypted by MI
DeïT-S
800 X * 32.9
31.6M
194.9G
5.6
DETR
ResNet- 101
800
42.5
60M
253G
5.3
YOLOS-B
DeïT-B
800
42.0
127M
538G
2.7
PEYOLOS on on images encrypted by MI
DeiT-B
800 * 39.5
128.2M
539.7G
2.5
be recognized by human eyes, while the comparison methods cannot. Moreover, RS and MI can be combined to further improve data security but at the expense of performance. Therefore, we conclude that PEViT-B achieves trade-off between performance and visual content protection.
4.3 COCO OBJECT DETECTION
The summary of the object detection results is shown in Table 2 with the competitive methods that contains roughly equal parameters. It can be observed that PEY- OLOS comparable to these methods. particular, our performance drop is only ~3.0% compared with the baseline method YOLOS, and only 2.8% compared with the CNN-based method Center- Net. Similar to PEViT, although PEYOLOS does not outperform the state-of-the-art methods, it is currently the unique model to achieve trade-off between performance and visual content protection for object detection.
which we compare PEYOLOS
4.4 PUZZLE SOLVER ATTACKER
Jigsaw puzzle solver, i.e., reconstructing the image from the set of shuffled patches, can be used to attack the images encrypted with MS. Since this problem is NP-hard Demaine Demaine (2007). solving jigsaw puzzles of non-trivial size is impossible. Most of the existing works in computer vision focus on the jigsaw puzzle problem composed of equally-sized image patches Cho et al. 2010); Pomeranz et al. (201)) Sholomon al. (2013); Paikin & Tal (2015), in which only pixels that are no more than two pixels away from the boundary adaptations how the image is decomposed, these methods might fail.
piece are utilized. Therefore, with
We choose the solver proposed in Paikin Tal (2013) to inveterate the effect attacks. The reason two-fold: (1) It is fast, aullacautomatic, and general solver, which assumes prior knowledge
8

---

Under review as a conference paper at ICLR 2023
Patch
Original images
Patch interval:
Patch interval: Drop ralio: 0%
Patzh inturval: Drop ralio: 0% Top
Patch interval:
Patch interval: 0
Patch interval: 0 Dcup rallo Top 73.3%
Deup ralio: 1%
Dzop ralio: 5%%
Drop ealio: 10% Top
Top
Top 77.6%
Top
Figure 5: Reconstructed images by the jigsaw puzzle solver proposed in Paikin Tal (013)). where the default image size is 224 224. Here, the effect of patch size. patch interval, and patch drop ratio on image reconstruction quality and classification performance is investigated.
about the original image and (2) It handle puzzles with missing pieces. We investigate the effect of patch size, patch interval, patch drop ratio, and image size on image reconstruction quality and classification performance. The results are shown in Figure [5] and Figure 4 We observe that increasing the number of patches (reducing the patch size or increasing the image size) the patch interval can enhance the strength of privacy protection the cost of performance. More importantly, even if 10% of the image blocks are dropped, the performance is only reduced by 1.6%, which allows users to drop patches containing sensitive information.
CONCLUSION
In this paper, we propose an efficient criaacy-presevvin learning paradigm that can destroy human- recognizable contents while preserving machine-learnable information. The key insight of our paradigm is to decouple the encryption algorithm from the network optimization via permutation- cquivariance, Two encryption strategies are proposed to encrypt images: random shuffling to a set of equally-sized image patches and mixing image patches that are permutation-cquivariiee ing ViT and YOLOS with minimal adaptations, they can be made (partially) coenuuttiioneqqivvrrin and are able to handle encrypted images. Extensive experiments on ImageNet and COCO show that the proposed paradigm achieves comparable accuracy with the competitive methods, meanwhile destroying buman-lecognizable contents.
By adapt-

---

Under review as a conference paper at ICLR 2023
REFERENCES
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. arXiv preprint (el1::667.64555 2016.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usumier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European Conference on Computer Vision, pp. 213-229. Springer, 2020.
Taeg Sang Cho, Shai Avidan, and William T. Freeman. A probabilistic image jigsaw puzzle solver. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 183-190, 2010.
E. Demaine and M. Demaine. Jigsaw puzzles, edge matching.and polyomino packing: Connections and complexity. Graphs and Combinatorics, 23:195-2083 2007.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei. Imagenet: A large-scale hi- erarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 245-255. Icee, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint 1XX1:20000111929 2020.
Yuxin Fang, Bencheng Liao, Xinggang Wang, Jiemin Fang, Jiyang Qi, Rui Wu, Jianwei Niu, and Wenyu Liu. You only look at one sequence: Rethinking transformer in vision through object detection. In Advances in Neural Information Processing Systems, pp.
2021.
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confi- dence information and basic countermeasures. Proceedings of the 22nd ACM SIGSAC Confer- ence on Computer and Communications Security, pp. 13221333, 2015.
Kai Han, An Xiao, Enhua Wu, Jianyuan Guo, Chunjing Xu, and Yunhe Wang. Transformer in transformer. arXiv preprint 1rXiv:2103.000 12, 2021.
Ali Hatamizadeh, Hongxu Yin, Holger Roth, Wenqi Li, Jan Kautz, Daguang Xu, and Pavlo Molchanov. Gradvit: Gradient inversion of vision transformers. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 10021-10030, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 16000-16009, 2022.
Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus). arXiv preprint 0rXiv:1606.08415. 2016.
Nandakumar Karthik, Ratha Nalini, Pankanti Sharath, and Halevi Shai. Towards deep neural net- work training encrypted data. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pp. 40-48, 2019.
Shen Li, Jianqing Xu, Xiaqing Xu, Pengcheng Shen, Shaoxin Li, and Bryan Hooi. Spherical con- fidence learning for face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15629-15637, 2021.
Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer back- bones for object detection. arXiv preprint a2Xiv: 16527, 2022.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
10

---

Under review as a conference paper at ICLR 2023
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv preprint arXiv:2103.14030, 2021.
Ziyao Liu, Jiale Guo, Kwok- Yan Lam, and Jun Zhao. Efficient dropout-resilient aggregation for privacy-preserving machine learning. IEEE Transactions on Information Forensics and Security, 2022.
Koki Madono, Masayuki Tanaka, Masaki Onishi, and Tetsuji Ogawa. Block-wise scrambled image recognition using adaptation network. In Workshop on Artificial Intelligence of Things (AAAI- WS), 2020.
Nikolay Malkin, Sameera Lanka, Pranav Goel, and Nebojsa Jojic. Studying word order through iterative shuffling. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 10351 10366, 2021.
Gonzalo Mena, David Belanger, Scott Linderman, and Jasper Snoek. Learning latent permutations with gumbelsinkhom networks. In International Conference on Learning Representations, 2018.
Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Intriguing properties of vision transformers. Advances in Neural Information Processing Systems, pp.
2021.
Genady Paikin and Ayellet Tal. Solving multiple square jigsaw puzzles with missing pieces. In Proceedings of the IEEE conference on computer vision and paltern recognition, pp. 4832-4839, 2015.
Dolev Pomeranz, Michal Shemesh, and Ohad Ben-Shahar. A fully automated greedy square jigsaw puzzle solver. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 16, 2011.
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. Designing network design spaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2020.
Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representa- tions. arXiv preprint arXiv: 1003.0:1880022111 2018.
Dror Sholomon, Omid David, and Nathan S. Netanyahu. A genetic algorithm-based solver for very large jigsaw puzzles. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1767-1774, 2013.
Koustuv Sinha, Robin Jia, Dieuwke Hupkes, Joelle Pineau, Adina Williams, and Douwe Kiela. Masked language modeling and the distributional hypothesis: Order word matters pre-training for little. In Proceedings of the Conference on Empirical Methods in Natural Language Processing,, pp. 28882913, 2021.
Warit Sirichotedumrong, Yuma Kinoshita, and Hitoshi Kiya. Pixel-based image encryption without key management for 2019.
deep neural networks. IEEE Access, 7:177844-177855
Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural net- works. In International Conference on Machine Learning, pp. 6105-6114 PMLR, 2019.
Masayuki Tanaka. Learnable image eneryption. In IEEE International Conference on Consumer Electronics-Taine pp. 1-2, 2018.
Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. arXiv preprint arXiv:2012.12811 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. arXiv preprint N.KV:1706.0362... 2017.
11

---

Under review as a conference paper at ICLR 2023
Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. arXiv preprint arXtv:2102.12122, 2021.
Ross Wightman.
Pytorch image models.
httpes//giihabb..mmmrviiigttttt
pytorch- mettorcchimmge-ee model s 2019.
Runhua Xu, James B.D. Joshi, and Chao Li. Cryptonn: Training neural networks over encrypted data. In International Conference on Distributed Computing Systems, 2019.
Runhua Xu, Nathalic Baracaldo, and James Joshi. srivacy-preservngg machine learning: Methods, challenges and directions. arXiv preprint wxXiv:2108.044111 2021.
Xingwei Yang, Nagesh Adluru, and Longin Jan Latecki. Particle filter with state permutations for solving image jigsaw puzzles. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2873-2880, 2011.
Qihang Yu, Yingda Xia, Yutong Bai, Yongyi Lu, Alan L Yuille, and Wei Shen. Glance-and-gaze vision transformer. In Advances in Neural Information Processing Systems, pp. 12992-13003, 2021.
Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Francis EH Tay, Jiashi Feng, and Shuicheng Yan. Tokens-to-tokee vit: Training vision transformers from scratch on imagenet. arXiv preprint 1 1986, 2021.
Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. In Advances in Neural Information Processing Systems, 2019.
12

---

Under review as a conference paper at ICLR 2023
A APPENDIX
The supplementary materials collect the source code package and the appendices on additional ex- periment results.
B IMAGE CLASSIFICATION
Attention map. We refer to "discriminative image patches" as the image patches that contain contents of the target category. To give an intuitive understanding of whether PEViT-B and PEViT-B with RPE can locate discriminative image patches, we provide Figure visualization results of attention maps from different layers of PEVIT-B, PEViT-B with RPE, and DeiT-B. It can be observed that (1) The attention of the class token of PEVIT-B, PEViT-B with RPE, and DeiT-B becomes more and more concentrated as the number of layers increases and (2) The discriminative image patches of PEVIT-B, PEViT-B with RPE, and DeïT-B are only partially attended by the class token at the lower layers. The reason for this might be that the context between image patches plays the key role in the image classification task, or the success of PEViT-B, PEVIT-B with RPE, and DeiT-B largely due their ability to model high-order co-occurrence statisties between image patches. Therefore, dropping out some patches has no significant impact on performance; see Figure
DeT8
TNTT
Figure 6: Attention map visualization results of the class loken in different layers of PEViT-B, PEViT-B with RPE, and DeiT-B for comparison.
Learned embedding filter visualization. further understand the distinctions between PEVIT- B, PEVIT-B with RPE, and DeïT-B, we provide in Figure 7 the top 48 principal components of the learned embedding filters. Each filter resembles plausible basis functions for a low-dimensional representation of the fine structure within each patch Dosovitskiy et al. (2020). It can be observed that the filters of DeiT-B have more structure than that of both PE PEVIT-B and PEViT-B with RPE. This means that DeiT-B focuses specific structures that contribute classification, while PEVIT- and PEVIT-B with RPE tend to focus more on the co-occurrence statistics between image patches. Figure shows the attention map of PEVIT-B, PEVIT-B with RPE, and DeiT-B from the first layer. The distribution of attention of both PEViT-B and PEViT-B with RPE is more dispersed than that of DeÎT-B. The reason for this might be that the co-occurrence statistics and PEViT-B with RPE due to their lack of perception of structural information contained in images. To some extent, this phenomenon is similar the conclusion in the natural language processing field that higher-order co- co-occurrence statistics of words play major role in learning Sinha et al. (202 Malkin al. (021), as discussed Sec. 1 of our main paper.
more important in PEVIT-B
Feature visualization. To give intuitive understanding of the distinctions between PEVIT-B, PEViT-B with RPE, and DeiT-B, we provide in Figure 9 the t-SNE feature visualization results of PEVIT, PEVIT-B with RPE, and DeiT-B. It is worth noting that both PEVIT-B and PEVIT-B with
13

---

Under review as a conference paper at ICLR 2023
RPE be generalized encrypted images that randomly shuffled. Interestingly, similar to DeîT-B, our PEViT-B and PEViT-B with RPE can still cluster the features belonging to the same class together while keeping the features belonging to different classes farther apart. This indicates the effectiveness of our permuration-equivariani designs on handling highly encrypted data.
(a) PEVIT-B
(b) PEViT-B with RPE
(c) DeiT-B
Figure 7: Learned embedding filter visualization. The top 48 principal components of the learned
embedding filters shown.
Images
(irput)
CALEBW ENO
PEVIT-B
(aayer
WE PAOO
PEVIT-B with RIE (layer
CALEGW
DaT-B (layer
GALEB
Figure 8: Attention map visualization results from PEViT-B, PEViT-B with RPE, and DeiT-B. The attention maps of the class token the first layer shown.
(a) PEVIT-B
(b) PEViT-B with RPE
(c) DeiT-B
Figure 9: I-SNE feature visualization of PEViT-B, PEViT-B with RPE, and DeiT-B. Features from 10 randomly sampled classes in the validation set are visualized.
Impact of RPE. The results of PEVIT with and without RPE are shown Table 3 can be observed that RPE can boost the performance of PEVIT. The motivation for designing RPE is as
14

---

Under review as a conference paper at ICLR 2023
follows. In PEViT, the positional encoding is removed due to its permutation-cquivaaae prop- erty. Through RPE, we illustrate that introducing positional embedding while retaining permutation- equivariant property is feasible, which might inspire other better designs of positional embedding that is permutaioo--quuiaaiinn.
Table 3: Impact of RPE on image classification performance on ImageNet.
Method
ImageNet top- acc.
PEVIT without RPE
78.7
PEVIT with RPE
79.7
15

---

Under review as a conference paper at ICLR 2023
C
OBJECT DETECTION
The input image of x W x C' is decomposed into batch of N patches with fixed resolution of P P. Then, for image patch, we further decompose the patch into a batch of sub-patches with a fixed resolution 2 x 2- Finally, these sub-patches are encrypted with mixing-up (MI), resulting highly encryted image that not huonn-becognizzble and is tough to be decrypted. PEYOLOS takes as input the encrypted images and outputs class and bounding box predictions for the original images.
Pipeline. The pipeline of PEYOLOS is shown in Figure 10
P P
Cls & BBox Predictions
MLP Heads
Par- Tok #1
Pat-Tok AZ
Pat- Tok A9
Der. Der-Tok #1
Det-Tok #2
Det- Tok #100
PEYOLOS
PE
PE
+ PE
PE
Pat-Tok #1
Par- Tok #2
Par-Tak e9
Dei-Tak #1
Dee-Tok
Det-Tok 4100
mixing
Non-Lincar Projection
Figure 10: Architecture overview of PEYOLOS.
Comparison results. The summary of more object detection results is shown in Table 4 in which we compare PEYOLOS with the competitive methods that contain comparable parameters. can be observed that (1) When using the backbone DeiT-Ti,, performance drop is only 3.4% compared with the baseline method YOLOS, (2) When using the backbone DeñT-S, our performance drop is only 3.2% compared with the baseline method YOLOS, and (3) When using the backbone DeiT-B, our performance drop is only 2.5% compared with the baseline method YOLOS. Similar to PEVIT, although PEYOLOS does not outperform the state-of-the-art methods, it is currently the unique model to achieve trade-off between performance and visual content protection for object detection; see Figure
Table 4: Object detection performance on the COCO test2017 7 dataset. FPS is measured with batch size 1 on single 1080Ti GPU. method
backbone
size
AP
params.
FLOPs
FPS
YOLOS-TI
DeiT-Ti
512 x 28.7
6.5M
18.8G
Deformable DETR
FBNet-V2 V3
800 X 27.9
12.2M
12.3G
35
YOLOSv4-Tiny
COSA
416 x 21.7
6.1M
7.0G
371
CenterNet
ResNet-18 18
512 X 28.1
129
PEYOLOS on images encrypted by MI
DeiT-Ti
512 X *
25.3
7.1M
19.0G
DETR
ResNet-18
800
36.9
29M
129G
7.4
YOLOS-S
DeiT-S
800 X 36.1
31M
194G
5.7
PEYOLOS on images encrypted by MI
DeiT-S
800 32.9
31.6M
194.9G
5.6
DETR
ResNet-101
800 X 42.5
60M
253G
5.3
YOLOS-B
DeiT-B
800 X 42.0
127M
538G
2.7
PEYOLOS images encrypted by MI
DeiT-B
800
39.5
128.2M
539.7G
2.5
16

---

Under review as a conference paper at ICLR 2023
It is worth noting that MI is not limited to YOLOS. With minimal adaptations, other object detec- tion frameworks based on plain Vision Transformer (ViT) can also be adapted to work on images encrypted by MI. Recently, it has been shown that, with ViT backbones pre-trained as Masked Au- toencoders, ViTDet Li et al. (022) can compete with the previous leading methods that were all based hierarchical backbones. By adapting the way the image patches are mapped like PEYO- LOS, PEViTDet can be readily obtained, achieving an AP of 41.2.
Qualitative results on COCO. To show the difference between PEYOLOS and YOLOS, we pro- vide in Figure 11 the qualitative results on COCO of both PEYOLOS and YOLOS. The main dif- ference between PE PEYOLOS and YOLOO is that PEYOLOS takes encrypted images as input while YOLOS take unencrypted images input. Compared with YOLOS, the performance degradation of PEYOLOS only 3%. Based on these observations, we conclude that the proposed learning paradigm can destroy human-recognizable contents while preserving machine-leamable informa- tion.
1ELES
feptt
Datccnn
(b)
Eps
(c)
(d)
PEYOLOS
Valas
Trlecto
(c)
(f)
Figure 11: Qualitative results on COCO with the backbone Deit-S. The main difference between PEYOLOS and YOLOS is that PEYOLOS takes highly eneryted images which are not human- recognizable and are tough to be decrypted as input while YOLOS take unencrypted images as input.
Attention maps of detection tokens. We inspect the attention maps of detection tokens that are related to the predictions. The experiments are conducted based on PEYOLOS and YOLOS, and the visualization results of the last layer are shown in Figure T2 and Figure 13 It can be observed that (1) For both PEYOLOS and YOLOS, different attention heads focus on different patterns different locations, (2) The corresponding detection tokens as well as the attention map patterns are usually different for PEYOLOS and YOLOS, and (3) Some visualizations are interpretable while others are not. The reason for this might be that both image patches and their context plays the key role in the object detection task and how the high-order co-oceurrence statistics between image patches affect the detection results still remains unclear.
17

---

Under review as a conference paper at ICLR 2023
Da Bosal
Det Searts
Na Pouaro
PEYOLOS (Durg)
YOLOS (Dacciad
Figure 12: The self-attention map visualization of the detection tokens and the corresponding pre- dictions, of the last layer attention heads based on PEYOLOS (Deit-S) and YOLOS (Deit-S).
20 (HH
PEYOLOS (Ours)
YOLOS (Baseline)
Figure 13: The self-attention map visualization of the detection tokens and the corresponding pre- dictions, of the last layer atttion heads based on PEYOLOS (Deit-S) and YOLOS (Deit-S).
18

---

Under review as a conference paper at ICLR 2023
Ablation on Patch Size for Object Detection. We have conducted ablation experiments the patch size for object detection, in which the backbone used is DeiT-Ti. The results are shown in Table 5 It can be observed that the best performance is achieved with the patch size of 16 x 16. Therefore, the patch size is set to 16 x 16 by default.
Table 5: The impact of patch size on object detection performance. Here, the backbone used is DeiT-Ti.
Patch size
Accuracy (AP)
20 20
20.7
16 16
25.3
14 14
23.4
10 x 10
23.2
Impact of nonlinear embedding. The results of PEYOLOS with linear embedding and without nonlinear emhedding are shown in Table It can be observed that nonlinear embedding can boost the performance of PEYOLOS. The reason for this might be that nonlinear embedding has stronger capacity, so it can capture more details useful for object detection.
Table 6: Impact of nonlinear embedding on object detection performance on COCO. Here, the backbone used DeiT-Ti.
Method
AP
PEYOLOS with linear embedding
23.9
PEYOLOS with two-layer nonlinear embedding
25.3
PEYOLOS with three-layer nonlinear embedding
26.2
D RECONSTRUCTION ATTACKER
We use recently proposed powerful Transformer-based framework, MAE He et al. (022) (tiny), to recover the original clean images from the images encrypted by MI. We adapt MAE with two modifications: (1) Patches are not dropped and (2) The linear patch embedding is replaced by nonlinear patch embedding, which is consistent with the patch embedding used in PEYOLOS. Here the nonlinear patch embedding is composed of two linear layers separated by GELU activation. Corresponding results are shown in Figure 14 We observe that: (1) The style reconstructed images is very different from original images and (2) Privacy-sensitive patches such faces and texts are blurred, and thus the reconstruction with MAE still cannot reveal the original identity of faces or the contents of texts. These observations indicate that recovering the original clean natural images from images encrypted by MI is a great challenge, demonstrating the effectiveness of MI regarding privacy preserving.
E TRANSFORMER-BANE BASED PUZZLE SOLVER ATTACKER
For the large-scale dataset ImageNet, the method in Mena et al. (2018) solves the jigsaw puzzle problem with 3 3 patches. Even in this simple case, the authors still face big challenges. For example, the authors claimed that "Learning in the Imagenet dataset is much more challenging, as there isnt sequential structure that generalizes among images".
We extend Mena et al. (2018) to the case where there are 14 x 14 patches (image size 224 x 224, patch size 16 X 16). We use the powerful pipeline of MAE He al. (2022) with the following proposed Mena et al. (2018), and (3) The positional encoding is removed, which necessary as the patch orders are randomly shuffled by RS. The results are shown in Figure 15]
modifications: (1) Patches are not dropped, (2) The loss function used is
can be observed that predicting the correct patch orders faces big challenges. The reason for this might be that (1) only a tiny fraction of 196! ~ 5 10363 possible patch orders are sampled, which difficult to estimate the underlying joint distribution of patch orders and patch content. Note that
19

---

Under review as a conference paper at ICLR 2023
sampling a large fraction of 196! z 5 10365 possible patch orders is infeasible in practice. (2) Extremely strong discriminative power are needed. For examples, we mirror an image, the patch orders should change accordingly. In this case, the imput patches are very similar, and model needs a very strong discriminative power to distinguish them.
Figure 14: Reconstruction attack on MI. Left: original images. Middle: images encrypted by MI.
Right: reconstructed images.
Figure 15: Reconstruction attack on RS. Left: original images. Middle: images encrypted by RS.
Right: reconstructed images.
20

---

Under review as a conference paper at ICLR 2023
To find the largest number of patches the reconstruction attack works, finetune pretrained ViT- base to solve the jigsaw puzzle in the settings of x 2, 3 x 3, x 4, and x patches. The training loss converges in the cases of x and 3 3 patches, but could not converge in all other cases. This suggests that the threshold 3 x 3: when the patch number is larger than 3 x 3, the reconstruction attack does not work any more (see Figure T6p. Our encryption is thus safe from ViT attack, where the patch number is 14 X 14.
Figure 16: Impact of patch number the reconstruction attack effect. Left: original images. Mid-
dle: permutated images. Right: reconstructed images.
21

---

Under review as a conference paper at ICLR 2023
F GRADIENT LEAKAGE ATTACKER
It has been shown that the training set will be leaked by gradient sharing Zhu et al. (2019). To evaluate the impact of gradient leakage on the security our method, we use the gradient leak- age attacker to recover the images encrypted by RS. The results are shown in Figure [7 It can be observed that (1) Gradient leakage attacker can restore images and (2) the restored images are encrypted. Therefore, we conclude that our paradigm does not prevent gradient leakage attacks, but can make the attacked images useless, thus protecting privacy.
Figure 17: Impact of gradient leakage on the security of our method. Left: original images. Middle:
images encrypted by RS. Right: recovered images.
G A QUANTITATIVE MEASURE OF PRIVACY LEAKAGE
We design simple measure to quantify the degree of privacy disclosure i.e., detecting faces original images and encrypted images and then considering the ratio of faces detected on encrypted images as measure, in which the face detection method used is MTCNN B
We have conducted experiments with this measure, and the results are shown in Table 7 can be observed that both RS and MI can protect the visual contents great extent.
Table Quantitative measure of privacy leakage. Please note that the experiments are conducted on the validation set of ImageNet.
Image source
of Detected faces
Ratio of leakage
Original images
15242
Encrypted image by RS Encrypted image by MI
1337
8.8%
212
1.4%
We also perform experiments to measure degree of privacy preservation by asking forty people to evaluate the effect of eneryption and attack 100 images (image size 224 x 224) and the results are shown in Table can be observed that (1) Both RS and MI can protect privacy well in most
22

---

Under review as a conference paper at ICLR 2023
cases and (2) The combination of RS and MI is hard to decrypt. In summary, would like to stress out that the larger the number of patches, the higher the security level and the overall encryption strength can be further improved by dropping sensitive patches, integrating MI, etc.
Table 8: Degree of privacy preservation evaluated by human beings.
Able to recognize image content beyond the label
Hard to recognize image content beyond the label
Method
Images encrypted by RS
0%
100%
(patch size 16 x 16)
Images encrypted by RS
0%
100%
(patch size
Images encrypted by
2%
98%
MI
Images encrypted by
0%
100%
RS + MI
Puzzle solver attack on RS
(patch size 16 x 16)
10%
90%
Sec, 4.4
Puzzle solver attack on RS
(patch size 8 x 8)
3%
97%
Sec. 4.4
Reconstruction attack on MI See. D in the appendix
6%
94%
Puzzle solver and reconstruction attack on RS + MI
0%
100%
23

---

Under review as a conference paper at ICLR 2023
H
IMPLEMENTATION DETAILS OF RS AND MI
The key implementation details of RS and MI are provided Algorithm aand Algorithm 2
Algorithm Pseudocode of RS PyTorch-lice like style.
Pd_chEmeeeed :odule)
f Key impl ementetion decails of RS
def _init_ _nnii_nnttt__ (self, patch_si7e, in_chans, embed_dim) auper Init_ patch_size to te_2tuple (patch_sizee) self.proj (In embad_dim, kernel_alzospatehhctt stride= pateh_aize)
def fozward x]:
x.shape
self.proj (x) flatten transppoee 2) X
shuffle (R):
sandom_idx taorch. (x.
shuffll. (idex] [zandom_idx, :1 un.uuquee (0))
torch. cat (shuffle, dim-C)
return
Algorithm Pseudocode MI a PyTorch-like style.
class MI PatchEbbeed (nn. Module):
Key impl emmntaaioon on decails of MT
impl implemented
fact i1.25: (4H Nam
[i-1) (4) (NX_11
init_ (selt, patch_size, in_chans, embed_dim); super (nnert)).iit..... init
satch_size = te_ (pacch___a c
embed_dim, kermel_size-paton_siree stride aelt.pool ná AvgPoo2222, stride-2)
psteh_size)
self an.GELU() self.proi_2 Convod (embod_dimg ambod_ kernel_aize=l, stride=1)
def fosward(self, x):
B,
x.shaps
x selt.proj_ 1o [x] x (878) (9/8)
self. (x) B (8/16) (X/16)
self.proj_2 tself (x) fletten transpooe f1, 2)
return
24


Reviews:
[{"review_id": "vTWT8mmH7l9", "reviewer": "Reviewer_ZYXS", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper tackles privacy-preserving image classification and object detection (in the context of vision transformers) based on random shuffling and mixing-up on the patch scale.\n", "summary_of_the_review": "With a much more rigorous privacy-preserving analysis and the justification of the MI encryption, the paper has a chance to get in.", "strength_and_weaknesses": "Strengths:\n\n1. Topic is important.\n2. Result is promising.\n\nWeaknesses:\n\n1. The primary issue is the lack of quantitative metrics to measure the degree of privacy preservation. Visual inspection of Figure 5 indicates that the main content has already been revealed. Although the authors emphasize that solving the jigsaw puzzle problem is NP-hard, in practice, we only need to find a not-so-good local optimal (e.g., in the mean squared error sense) that is sufficient to reveals the main semantics w.r.t. the task at hand.\n\n2. If the reviewer understands correctly, after the MI encryption process, even the encrypters themselves cannot perfectly recover the original image (see Eq. (2)), which is unfavorable. We expect encryption and the corresponding decryption together to form a bijection.\n\n3. For image classification, we tend to rely on permutation-equivariant and permutation-invariant operations as suggested by the authors. Then, what is the motivation to incorporate a reference-based positional embedding in Eq. (10)?\n\n4. The math notation in Eq. (12) is less clear. Specifically, what is $\\mathbf{x}_i^\\mathrm{DET}$?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "teEkaNVi7o", "reviewer": "Reviewer_rSs2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a privacy-preserving training scheme that consists of two parts: encryption strategies based on permutation-equivariance and (partially) permutation-equivariant learnable network.\nThe contributions of the paper are the two encryption strategies, names Random Shuffling and Mixing, together with the modifications of the existing transformer-based image classification and object detection to learn from encrypted images.", "summary_of_the_review": "In summary, the paper still has some valuable contributions in terms of encryption strategies. Thus, my recommendation rating for the paper is weak accept.", "strength_and_weaknesses": "Strength:\n1. The overall paper is well-written and defined. It is easy enough to follow and reproduce.\n2. Extensive experimental results to demonstrate the strength of the proposed method.\n\nWeaknesses:\n1. Limitation of the paper is not discussed. \n2. The change to the existing method so that it can learn from encrypted images is not significant. For example, the change from the existing YOLOS to the proposed PEYOLOS is the input x^p to x_s as in Eq. 12.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "nNH-MRQv0Jj", "reviewer": "Reviewer_Y75t", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes RS and MI, two permutation-based image encryption algorithms designed to render images unrecognizable to humans, yet still enable image classification and object detection respectively, by downstream vision transformers. The first step of both encryption algorithms is to partition images in to N equally sized patches. Next, RS, short for “random shuffling”, removes the positional embedding of the patches and shuffles their order, leaving a “bag of words” style input. MI, short for “mixing-up”, partitions each of the N patches into M sub-patches and sets each sub-patch equal to the mean of the M sub-patches. The encrypted images are then fed to \"permutation-equivariant\" variations of ViT and YOLOS, referred to as PEViT and PEYOLOS for image classification and object detection respectively. Compared to ViT, PEViT replaces the conventional positional embedding with a novel reference-based positional embedding. Compared to YOLOS, PEYOLOS replaces the linear patch embedding with a two-layer non-linear patch embedding. Experimental evaluations on ImageNet and COCO demonstrate that the proposed methods achieve near SOA performance despite operating on encrypted inputs. ", "summary_of_the_review": "The main idea, albeit simple, is interesting and novel and seems to produce good results. However, the evaluation is lacking in that privacy is not defined and is missing key ablation studies.", "strength_and_weaknesses": "# Strengths\n\n* The main idea of the paper of combining permutation-based image encryption algorithms with transformers is interesting and novel.\n\n* Experimental evaluations on ImageNet and COCO demonstrate that the proposed methods achieve near SOA performance despite operating on encrypted inputs.  \n\n\n# Weaknesses\n\n* The paper fails to provide a concrete definition of privacy. For example, the paper claims that the “proposed paradigm can destroy human-recognizable contents”, but never specifies what is meant by this.\n\n* Given that the stated aim of the proposed encryption algorithms is to “destroy human-recognizable contents”, humans should play some role in the evaluations. Where are the human studies? \n\n* The paper is missing ablation studies for the novel transformer architectures. Where are the experiments demonstrating the need for the proposed RPE in PEViT and the nonlinear embedding in PEYOLOS?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "AJloFOsKoaQ", "reviewer": "Reviewer_E1ZJ", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to \"encrypt\" images by using two techniques. First one (RS) is to cut each image into patches and then reshuffle the order of tiles. Second one (MI) is to mix up some of the patches.\nAuthors show that vision transformer without positional embeddings (PEViT) can successfully learn to recognize images encoded by RS method. And they also show a modification of object detector YOLO which can perform object detection on images encoded by MI method.", "summary_of_the_review": "Strong reject.\n\nPaper claim to propose an encryption scheme which protects privacy. However authors neither follow any common privacy definition (for example, differential privacy), nor they define their own notion of privacy. In addition, evaluation of encryption properties of the proposed methods is lacking.", "strength_and_weaknesses": "Strength:\n1) Paper is reasonably well-written and easy to understand.\n2) It is an interesting observation that vision transformer does not need positional embedding for successful image recognition. Idea of RLE is also interesting.\n\nWeaknesses:\n1) Authors claim \"encryption\" and \"privacy protection\", however they don't define what they mean by privacy and don't define their threat model. One of the adopted notions of privacy for ML models is differential privacy ( https://arxiv.org/abs/1607.00133 ). It's fine to define and use different notion of privacy, but it has to be well defined. It seems like, by privacy authors imply inability for a human observer to recognize what's on the image. However this is highly subjective, moreover proposed methods arguable don't even fulfill this goal (see below).\n2) Lacking evaluation of reconstruction attack. I didn't find any numerical evaluation. Evaluation in experimental section is summarized by a quote: \"As shown in Figure 1, the visual contents of encrypted images are nearly-completely protected from recognizing by human eyes\". For the reference, figure 1 shows a few images before and after encoding. So it appear to me that authors claim that solving jigwas puzzle is a hard problem, showing few images before and after encoding and claim that this is the reason why their method protects privacy.\n3) Authors over estimate complexity of solving a jigsaw puzzle. The experiments are done in a setting when RS method split image into 14x14 patches. Authors refer to a paper about automated jigsaw puzzle solver from 2015 and claim that it's a sufficiently hard problem.\nArguable, individual 14x14 jigsaw puzzle (196 pieces) could be easily solved by human in a few hours at most. Which means that any individual image encoded with RS method does not really \"destroy human recognizable content\".\nFor comparison, no human can manually decrypt a file, which is encrypted with AES ( https://en.wikipedia.org/wiki/Advanced_Encryption_Standard )\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Decision: Reject