Private Image Reconstruction from System Side Channels Using Generative Models
Authors: Yuanyuan Yuan
Shuai Wang
Junping Zhang
Venue: ICLR 2021 Poster (2021)
Keywords: side channel analysis

Abstract: System side channels denote effects imposed on the underlying system and hardware when running a program, such as its accessed CPU cache lines. Side channel analysis (SCA) allows attackers to infer program secrets based on observed side channel signals. Given the ever-growing adoption of machine learning as a service (MLaaS), image analysis software on cloud platforms has been exploited by reconstructing private user images from system side channels. Nevertheless, to date, SCA is still highly challenging, requiring technical knowledge of victim software's internal operations. For existing SCA attacks, comprehending such internal operations requires heavyweight program analysis or manual efforts.

This research proposes an attack framework to reconstruct private user images processed by media software via system side channels. The framework forms an effective workflow by incorporating convolutional networks, variational autoencoders, and generative adversarial networks. Our evaluation of two popular side channels shows that the reconstructed images consistently match user inputs, making privacy leakage attacks more practical. We also show surprising results that even one-bit data read/write pattern side channels, which are deemed minimally informative, can be used to reconstruct quality images using our framework.

Published as a conference paper at ICLR 2021
PRIVATE IMAGE RECONSTRUCTION FROM SYSTEM SIDE CHANNELS USING GENERATIVE MODELS
Yuanyuan Yuan & Shuai Wang"
Junping Zhang
Department of Computer Science and Engineering The Hong Kong University of Science and Technology Clear Water bay. Hong Kong SAR {yyuanaq, shuaiw] Ocse ust hk
Department of Computer Science
Fudan University
Shanghai, China jpzhang fudan edu. en
ABSTRACT
System side channels denote effects imposed on the underlying system and hard- ware when rumning program, such as its accessed CPU cache lines. Side chan- nel analysis (SCA) allows attackers to infer program secrets based on observed side channel logs. Given the ever-growing adoption of machine learning service (MLaaS), image analysis software on cloud platforms has been exploited by reconstructing private user images from system side channels. Nevertheless, to date, SCA is still highly challenging, requiring technical knowledge of victim software's internal operations. For existing SCA attacks, comprehending such internal operations requires heavyweight program analysis manual efforts.
This research proposes attack framework to reconstruct private user images processed by media software via system side channels. The framework forms an effective workflow by incorporating convolutional networks, variational autoen- coders, and generative adversarial networks. Our evaluation of two popular side channels shows that the reconstructed images consistently match user imputs, mak- ing privacy leakage attacks more practical. We also show surprising results that even one-bit data read/write pattern side channels, which are deemed minimally informative, can be used to reconstruct quality images using our framework.
INTRODUCTION
Side channel analysis (SCA) recovers program secrets based on the victim program's nonfunctional characteristics (e.g., its execution time) that depend on the values program secrets. SCA consti- tutes a major threat in today's system and hardware security landscape. System side channels, such as CPU cache accesses and operating system (OS) page table accesses made by the victim software, widely used recover program secrets under various real-world scenarios (Gullasch al. 2011 Aciiemez & Koc 20066] Wu et al. 2012 Hähnel et al.) 2017; Xu et al.) 2013) Yarom et al. (017).
To conduct SCA,, attackers first conduct online phase to log trace of side channel data points made by the victim software (e.g., its accessed CPU cache lines). Then, attackers launch an offline phase to analyze the logged trace and infer secrets (e.g.. private inputs). Enabled by advances in system research, the online phase can be performed smoothly (Xu et al.] (015).. Nevertheless, the offline phase challenging. requiring comprehension of victim software's input-relevant operations and how such operations influence side channels. The influence is program-specifie and obscure (see an example in Fig. [T. Even worse, side channel data points made by real-world software are usually highly noisy. For instance, executing 1 lib jbej (libjpeg] (020) to decompress one unknown JPEG image produces trace over 700K side channel data points, where only a small portion depends on the image content. Identifying such input-dependent data points from over 700K records is extremely difficult.
Launching SCA to recover images processed by media software constitutes common threat in the era of cloud computing (Xu et al. 20155 Hähnel et al.) 2017). especially when machine learning as service (MLaaS) is substantially offered (e.g., for face recognition). When envisioning the high
"Corresponding Author

---

Published as a conference paper at ICLR 2021
risk of violating user privacy, there is demanding need to understand the adversarial capabillity of reconstructing private images with SCA. To date, the offline inference phase of existing SCA attacks requires lots of manual efforts with heuristics (Xu et al.) 2015: Hähnel et al. 2017). While some preliminary studies explore to use Al models to infer secrets (Hospodar et al. 201 I Kim et al. 2019) primarily driven by classification, i.e., predicting whether a particular bit of crypto key is O or 1. In contrast, reconstructing user private images requires to synthesize and enhance images from a more holistic perspective.
Cagli al.] 2017; [Hetwwe al. 2018), their approaches
Recent advances in generative models, such as generative adversarial network (GAN) and variational autoencoder (VAE), have enabled major thrust in image reconstruction. given subtle signals even cross-modal settings, e.g., voice-to-face or text-to-image -on-tet-mmmeeeee (Radford et al. [016;) Reed et al. [016]] Wen et al., 2019 Hong et al. 2018b). Inspired by this breakthrough, we propose an SCA framework using generative models. Given trace of side channel data points made by image analysis software (e.g., lib jpeg) when processing a user input, we reconstruct an image visually similar to the input. Each logged side channel trace, containing around a million records, is first encoded into matrix and pre-processed by a convolutional neural network (CNN) for feature extraction. Then, VAE network with a learned prior (referred to VAE-LP) is employed to reconstruct an image with a holistic visual appearance. We further supplement VAE-LP with a GAN model to enhance the recovered image with vivid details. The GAN generator yields the final output.
Our attack exploits media libraries, jibj (lijjeeg 2020) and uPNG (Middleditch) 2010), us- ing two popular side channels, CPU cache line accesses and OS page table accesses. Our attack is independent of the underlying computing infrastructure (i.e., OS, hardware, image library imple- mentation). We require enough side channel logs for training, which is consistently assumed by previous works (Heuser& & Zohner 2012) Maghrebi et al. 2016). While existing attacks particularly target 1b jpeg and leverage domain knowledge, system hacking, and manual efforts to infer pixel values (Xu et al. 2015 Hähnel et al. (017), we show that images with many details can be recon- structed in an end-to-end manner. We also show surprising results that enabled by our framework, side channel traces composing one-bit data read/write patterns, which prima facie seems minimally informative, suffice recovering images. We conduct qualitative and quantitative evaluations on spe- cific and general datasets representing daily images that can violate privacy if leaked. The recovered images manifest consistent visual appearances with private inputs. The recovered images also ex- hibit high discriminability: each recovered image (e.g., face) can be matched to its reference input among many candidates with high accuracy. In summary, we make the following contributions:
At the conceptual level, we present the first generative model-based SCA. Our novel approach learns how program inputs influence system side channels from historical side channel logs to construct user private images automatically. We, for the first time, demonstrate surprisingly effective attacks toward even low-resolution side channels like one-bit data read/write access patterns.
At the technical level, we design an effective framework by incorporating various design principles to facilitate image reconstruction from side channels. Our framework pipelines 2D CNN, VAE-LP, and GAN models to systematically enhance the quality of generated images.
At the empirical level, our evaluations show that the proposed framework can generate images with vivid details and are closely similar to reference inputs. The reconstructed images show high discriminability, making privacy leakage attacks more practical.
This is the first paper to conduct SCA with generative models,, revealing new SCA opportunities and unknown threats. Our code is at httt:s///gituub..cmm//ee////////
BACKGROUND
To formulate SCA, let the attacked program be P and its input domain be For a deterministic and terminating program P. the program execution can be modeled as mapping P 7 F where E represents program runtime behavior (e.g., memory access). As common assumption (Hähnel ct al. 2017), program inputs are private and profitable for attackers. Since different inputs i, e can likely induce different e, e' E E, using input-dependent e E E enables to infer i.
Modern computer architectures have primarily zeroed the possibility for adversaries to log E E. Nevertheless, an attacker's 's view on P can be modeled as a function view E - O that maps E to side channel observations O. Hence, the composition (view o P) : - O maps inputs to side channel data points that can be logged by attackers. The view indicates the attacker's ''s capability, and for typical system security scenarios, the view is formulated as view Emmm Ocache
2

---

Published as a conference paper at ICLR 2021
VM Instance
VM Instance
it (i 1)
cache 1ine 0x7EF10
private
imputs
memoryy
Image library
Malicious app
recordingg
array 24017 else
//acceas cache 1ine OxTrr
cache 1ine 0x7
Preserve scenet-dspendent side channel data points
accesss
//aseess cache line 0xY1202
recording
CPU cache line
atray[59];
cacne line Ex/1110
Page table entry
endfor
Data read/write patliern
Faccess cache 1ine 0x7EE02
actay[60];
Infer socret as I
Malicious Host OS/Hardware
(a) SCA toward cloud platforms to recover private inputs side trace recording is straightforward.
(b) Victim code snippet with 3 memory uccesses different locations.
(c) Offline phase to infer the logged side channel trace.
Figure 1: Performing SCA to exploit cloud computing platforms.
Opages where Emem denotes a trace of accessed memory locations when executing P with i, and Oeache and O page represent CPU cache and OS page table side channels, respectively. Despite being unable to monitor Ememm attackers can log accessed cache lines Oeacke or page table entries O Opage derived from E.mem_ Attackers then infer Emem and recover i. We now concretize the procedure by introducing how SCA is used to exploit cloud platforms in a two-step approach as follows:
Online Phase to Record O. Considering a cloud environment in Fig. [[a]] where two users, one normal and one malicious, deploy two virtual machine (VM) instances the host. Private images i E T uploaded by users are processed by media library within the left VM. Modern computer design, e.g., Intel SGX (Intel 2014), guarantees that i / and the execution of / cannot be viewed from outside the VM. However, when processing i, P usually imposes a large volume of CPU cache and page table accesses, which, as shown in Fig. ((a) can be recorded by the co-located malicious VM or the malicious host OS in fully automated manner (Han et al. 2017: Chiang et al. 2015
Liu et al. 2015a Xu et al. 2015 Hähnel et al.) 2017).
Offiine Phase to Infer i. Once side channel traces o E O are collected, an offline phase is conducted
to infer (view o P)-1 : O - and recover i. Fig. 1(b) presents sample code, where depending on
values of input i, different memory locations (and cache lines) will be visited. Fig. [[c] shows the corresponding trace of logged cache side channel records. To infer i, attackers eliminate the second record (since it snut-independent) and infer as according to the first record.
Attackers anticipate to 1) pinpointing subset of records o* S that depend on i, and to 2) recov- ering the mapping from o* to i. However, real-world side channel traces (e.g.. generated by uPNG) could contain over one million records, where only tiny portion o" is input-dependent Even worse, constructing the mapping between and o" requires a deep understanding of program control flows (e.g., how i affects program execution and induces cache accesses in Fig. [(b). To date, these tasks require either manual effort (Xu et al. 2015 Hähnel et al. 2017) or formal analysis (Doychev et al.) 2013 Wang et al. 2017 2019), which program-specific and error- error-prone prone with low scalability.
Existing research tackles the offline phase challenge by proposing profiling-basee SCA (Maghrebi al. 2016 Hettwer et al. 2018 Kim et al., 2019), where models are trained to approximate (viewo However, existing work focuses on predicting particular bits of crypto keys from succinct side channel traces, e.g., few hundred records (Hettwer et al., 2020). In contrast, this is the first work shows that by incorporating generative models, SCA can be conducted to exploit real-world media libraries and holistically reconstruct high-quality and discriminable images.
P)- :
THE PROPOSED FRAMEWORK
A common assumption shared by SCA (Heuser & Zohner 2012 Hähnel et al. 2017 Xu et al.)
2015) is that the attackers can profile the victim software locally or remotely with training inputs and collect corresponding side channel traces. We train a model to learn how different inputs can influence side channel traces. Then, given a side channel trace logged when processing an unknown image, our framework reconstructs image that is visually similar to the unknown input.
Our framework has two pipelined modules (see Fig. D. Given a side channel trace T; corresponding to processing an image we first encode T; into matrix. The encoded matrix will be fed to the
VAE-LP module to generate image teraces and further use GAN to denoise isace and yield the final output ÂGAN. We now elaborate each module. More details are given in Appendix
3

---

Published as a conference paper at ICLR 2021
Trace Eneoder
Decoder
Encoder
Decoday
mairix enoading
Rou
Dus
Image Encoder
Powe
Generator
sampling
Pake
Real
Diseriminator
(a) VAE-LP. LP. The training plaso takes side channol traces (T) and reference images (i) inguts. The predication plase only retains the upper dataflow marked in blue.
(b) module nccepes an output the VAF-LP module and yenerates the final ourput framework.
Figure 2: The proposed generative framework for recovering private images from side channel traces.
3.1 SIDE CHANNEL TRACE ENCODING
Real-world software is highly complex, and processing one image could generate a huge amount of records, where only few records are secref-dependent. Processing overlong traces for previous attacks is very difficult and requires considerable domain-speciffi knowledge, expertise, and even manual efforts to locate and remove irrelevant records (Xu et al. 2015; Hähnel et al. 2017).
Despite the general difficulty of processing overlong traces (each trace contains about 700K to 1.3M data points in evaluation), we note that adjacent records side channel trace are often derived from the same or related modules (e.g., functions) of the victim software. Hence, we "fold" each side channel trace into N N K matrix to approximate spatial locality which exploited by CNNs. A trace is first divided into K segments, where N adjacent points in a segment are put into one row, in total N rows. We do zero padding. CNNs are deployed in the trace encoder of VAE-LP to process the encoded matrices. Overall, we have no assumption of the access pattern or the convolutional structure of the inputs. Side channel traces are generally sparse, where only a small portion is private-related (see Appendix]foor experimental information). To smoothly process the side channel traces with generative models, we thus employ CNN models to pre-process, side channel traces.
be further
3.2 THE VAE-LP MODULE
VAE-LP extends standard VAE by replacing its fixed Gaussian prior with a learned prior (Den- ton & Fergus 2018), which represents the latent distribution of side channel traces. VAE-LP is trained using both real-life images and their corresponding side channel traces. By incorporating the side channel trace encoder, can extract latent representations from the logged side channel data points. Simultaneouslyy, by integrating corresponding reference images during training, we provide guideline to help the image decoder to generate quality images. As shown in Fig. 2(a) the trace encoder EnCtrace (marked in blue) employs 2D CNNs and can extract features from side channel traces T; in the encoded matrices. The output of Enctrace constitutes the learned prior distribution of latent variable, namely p( p(2r,) ), rather than fixed Gaussian distribution that VAE usually is. The decoder Dec takes the mean of p(æT;) ) as its input and outputs the image generated from side chan- nel traces. The training phase also employs the image encoder Encimage (marked in red), which accepts reference images i and outputs q22,(i).
We train the VAE-LP network by performing forward propagation separately for two different data resources. We then use two generated images to compute reconstruction loss and perform one iteration of backward propagation. Let êtrace and Dectrace be the generated image and decoder Dec by conducting forward propagation with only T,. Similarly, let ismage and DeCamage be the generated image and Dec by conducting forward propagation with only i. Parameters of Dectrace and DeCimage are shared in the training phase, and the loss of the VAE-LP module is defined as follows:
Lossy AE-LP L1 [i, Rimage) L Li +
where three terms, namely, i) reconstruction loss Li (i, (mmge)) derived from the reference input, ii) a reconstruction loss T. (i, Ssrace) derived from the side channel trace, and iii) KI--divergence
4

---

Published as a conference paper at ICLR 2021
Table 1:
Side channels derived from memory access made by victim software using address addr.
Side Channel Name
Side Channel Record Calculation
CPU
Cache Index OS Page Table Index
addr R L where L, denoting cache line size, is on modern X86 architectures.
oddr (~~ M) where denoting PAGF_MASK, is 4095 on modern x86 architectures.
Data Read/Write Pattern
bit (0/1) denoting whether this memory access read or write operation.
that forces q(z/(t) to be close to p(zr.) are subsumed. During the generation phase, we remove EnCimage and DéCtrace- Enctrace and DeCsmage retained to yield israce given logged T; The generative process is given by:
21, p(zr.) itrace p(ilzr,)
3.3 THE GAN MODULE
VAE-LP module is seen to recover êerace with relatively coarse gorsse-rained information (sce Sec. 5.1 As will be claborated in Sec. 4 different private inputs can manifest identical side channel patterns. Hence, some details limitations, we further deploy a GAN module (see Fig. ((b) which takes the output of the VAE- LP module, itraces and generates the final output iGAN To smoothly refine itraces we employ an autoencoder as the generator G of GAN. The loss of the extended GAN model defined as follows:
inevitably missing during input reconstruction. To tackle this inherent
LOSSGAN 7 (GL(Grrrraae) itrace) Ei-p(i) (log D(i) +E; B P (ebraac)) [log(1 D(G(irrace)) )))l
Compared with standard GAN, we extend the loss function with I loss itrace and G(itrace) with a weight of y to force G to retain the holistic visual appearance delivered by âzrace- L loss generally acknowledged to perform better on capturing the www-frquency part an image (Isola et al. (017)). Indeed, our evaluation shows that L1 loss, as common setting, sufficiently conducts SCA and recovers user private inputs of high quality.
ATTACK SETUP
As introduced in Sec. 2 popular system side channels are primarily derived from program memory accesses. Let addr be the address memory location accessed by the victim software, Table reports three utilized side channels and how they derived from memory accesses. Cache line and page table side channels are commonly used for exploitation (Hähnel et al., 2017 Yarom & Falkner 2014). Furthermore, enabled by our framework, a very low-resolution side channel of data read/write access patterns can, for the first time, be used to reconstruct high-quality images. Fig. holistically depicts how attackers can monitor cache and page table side channels. Data read/write patterns can be similarly recorded by monitoring how caches or page tables are visited.
Table that different addr can be mapped to the same side channel record. Similarly, different inputs can induce identical memory address addr. For instance, in Fig. [(b) array [ (59] and array [60] will always be executed as long * 1. Two layers many-to-one mapping amplify the uncertainties of synthesizing discriminable images of high quality. It is easy to see that we are not simply mapping a trace back to an image.
Each memory address addr has 48 bits, denoting large range of values. We normalize the memory address value (discrete integers) into continuous values within [0, 1]. Overall, while arbitrary 48-bit integers have large range, side channel data points indeed vary within small range. For instance, for cache based side channels, the possible values are limited by the total number of CPU cache units. In all, side channel data points large values ranging in relatively small range.
Attack Target. We attack two media libraries, 1 lib jpeg and uPNG, to reconstruct private user images of JPEG and PNG formats. Previous image reconstruction SCA (Xu al., 2015: Hähnel al. 2017) only exploit libipeg. PNG and JPEG denote very popular image compression stan- dards, and given an image in JPEG/PNG format. libipeg and uPNG can reverse the compression to generate bitmap image as the basis of many image analysis tools, e,g., the Python Pi Pillow li- brary (Clark (020)). The decompression process introduces many input-dependent memory accesses which, from the attacker's perspective, can be reflected side channels according to Table
5

---

separate task, Published as a conference paper at ICLR 2021
VAE-LP
VAE-LP & GAN
VAE-LP
VAE-LP&
VAE-LP
VAE-LP & GAN Reference Inpot
CelebA
LSUN
KTH
Page
Cache
Figure 3:
Qualitative evaluation results of three datasets in terms of three different side channels. Correspond- ing reference images (i.e., private user inputs) are presented on the rightmost column for comparison.
We share common assumptions with existing profiling- based SCA (Hospodar al. 2011 Heuser & Zohner 2012) that side channel traces have been well prepared for use. For our experiments, we use 2005), runtime monitoring tool, intercept memory accesses of victim software. A logged memory access trace will be converted into three separate side channel traces according denoting the attacker's view on Each side channel trace generated by libipeg or uPNG contains 700K to 1.3M records. See Appendix A for attack setup details. We evaluate traces logged via different side channels separately. Evaluating the composition of side channels (i.e., "mega-side channel")) is not aligned with how real-world SCA is typically launched.
Pin (Luk
Table [
5 EVALUATION
We present the first systematic approach to reconstructing images from side channels. There is no previous research for empirical comparison. Two closely related works provide tools for use (Xu et al. 2015] Hähnel et al., 2017). As disclosed their papers, manual efforts are extensively used reconstruct images. For instance, both methods treat image color recovery by iterating multiple reconstruction trials and manually picking one with relatively better visual effect. Xu al. (205)) exploit page table side channels and colors rarely recovered. [Hähnel] et al. (217)) recover adequate image colors but only exploit finer-grained cache side channels. Also, domann-pecific knowledge on 1 libjpeg is required locate tiny portion of dereeddeeendent side channel data points for use. In contrast, we present an end-to-end approach to recovering colorful images with high quality, by directly analyzing page table or cache side channel trace of up to 1.3M records. Our attack treats victim software (libjpes or uPNG) as "black-box" (no need for source code) and is independent of any underlying computing infrastructure details.
Benchmarks. Three datasets are primarily used in the evaluation, containing typical daily images that could violate privacy if leaked to adversaries. Consistent with existing research reconstructing images from audio recording (Wen et al. 2019 Oh al. 2019), we accelerate the model training using images of 3 x 128 x 128 pixels. Wen et al. (2019) use images of an even smaller size (3 x 64 64). See Appendix B for model implementation and training details.
(i) Large-scale CelebFaces Attributes (CelebA) (Liu et al. 2015b) contains about 200K celebrity face images. We randomly select 80K images for training and 20K images for testing.
(ii) KTH Human Actions (KTH) (Laptev Lindeberg 2004) contains videos of six actions made by 25 persons in 4 directions. For each action, we randomly select videos of 20 persons for training and use the rest for testing. We have 40K images for training and 10K images for testing.
(iii) LSUN Bedroom Scene (LSUN) (Yu et al., 2015) contains images of typical bedroom scenes. We randomly select 80K images for training and 20K images for testing.
5.1 QUALITATIVE EVALUATION RESULTS
Fig. shows the reconstructed images in different settings. In addition to reporting the final outputs (i.e.. the "VAE-LP & GAN" column), also report images generated only from the VAE-LP module in the "VAE-LP" column for comparison. More results are given in Appendix For most of the cases, the reconstructed images and reference images show consistent visual appearances,

---

Published as a conference paper at ICLR 2021
Table 2: Discriminability evaluation. We present success rates of random guess (RG) for the comparison.
==11 N =
100
= 5.N =
100
20 N
=
100
CelebA
K"H
GSON
Ce TeDA
RTF
5SUN
RG
Ce
RTH
LSON
CPU Cache Line
47.28%
38.22%
10.00%%
18:
74.98%
63.56%
28.74%
5%
90.70%
83.72%
58.88%:
20%6
OS Page Table
16.16%
35.86%
2.26%
1%
37.96%
61.28%
8.62%
5%
65.14%
82.22%
26.42%
20%
Read/Write Partern
19.58%
33.44%
2,24%
42.44%
8.28%
5%
67.82%
80.14%
26.649:
== 1
5000
6.N
500
20 NV
300
Callba
K"H
T-SUN
Go 2elenA
NSUN
R5
G6 100A
F.SUN
CPU Cache Line
28.68%
24.30%
3.04%
0.28%
52.22%
41.24%
10.90%
15%
72.98%
61.08%
25.92%%
4%
Page Table
6.56%
22.58%
0.46%
17.90%
38.80%
19
35.22%
58.90%
Reul/Write Pattern
8.42%
20.64%
0.50%
0.2%%
21.18%
36.22%
2.54%
19
39.84%
35.74%
7.08%
4%
such as gender, skin color, bedroom window, and human gesture. Images in the LSUN dataset contain many subtle bedroom details, imposing relatively higher challenge for reconstruction.
Realistic and recognizable images can be recovered using cache side channels (especially for LSUN) while images recovered from the other side channels are relatively blurry. As explained in Table T cache line indices (addr >>> 6) are closer to the memory address addr (only missing the lowest bits), while page table indices eliminate the lowest 12 bits from addr (typically lower bits in addr are informative and likely influenced by inputs), and each read/write pattern has only one bit.
Compared with images generated by VAE-LP, the GAN module enhances the image quality by adding details and sharpening the blurred regions. GAN may overly enhance the image quality (e.g., the first LSUN case with jungle green wallpaper). However, GAN indeed vital to exploit user privacy. For example, considering the first human gesture case in KTH, where the image recon- structed from cache side channels contains a "black bar" when using VAE-LP. The GAN module enhances this obscure image and reconstructs the human gesture, thus violating user privacy.
5.2 QUANTITATIVE EVALUATION RESULTS
To assess the generated images w.r.t. discriminability, we first study the accuracy of matching a reconstructed image i to its reference input é. To do so, we form a set of N images which include the reference input i and - images randomly selected from our testing dataset. We then measure whether i appears in the top-k most similar images of i. Conceptually, we mimic de-anonymizattion attack of user identity, where N scopes the search space attackers are facing (e.g., all subscribers of a cloud service). We use a perceptually-basee similarity metric, SSIM (Wang et al.) (004)), to quantify structural-level similarity between the reconstructed images and reference inputs.
Table reports the evaluation results of six practical settings. Consistent with Fig. 3 cache side channels help reconstruct with better discriminability (highest accuracy for all settings in Ta- ble LSUN images have lower accuracy. As shown in Fig. 3 images in LSUN contain many subtle bedroom details and deem challenging for discrimination. We achieve the highest accuracy when k = 20 and N = 100, while the accuracy, as expected, decreases in more challenging settings (e.g., when and N 500). Evaluations consistently outperform the baseline random guess. For instance, while the accuracy of random guess when k = and N = 500 is 0.2%, we achieve higher accuracy (range from 0.46% to 28. (.6%) across all settings. Appendix also conducts this evaluation using images generated from only VAE-LP. We report that better discriminability can be achieved for all datasets when supplementing VAE-LP with GAN.
For face images in CelebA, we also study how well different facial attributes are being captured in the reconstructed images. We use Face++ (fac) 2020). a commercial image analysis service, to classify reconstructed images and reference inputs w.r.t. age and gender attributes. Fig, reports the confusion matrices for age and gender attributes and distributions of training data for the reference. The reconstructed images are produced using cache side channels. Overall, achieve a good agreement for both male and female labels. We also observe correlated classification results for most age groups. The age distribution indicates that early adulthood (20-40) and middle age (40- 60) mostly dominate the dataset, which presumably induces biases in the age confusion matrix. Similarly, "male" has smaller representation the training set, potentially explaining its lower agreement in the confusion matrix. Appendix D also conducts this evaluation using only VAE-LP or using other side channels where comparable results can be achieved.
7

---

Published as a conference paper at ICLR 2021
Output
Gender Distribution of Training Images
Age Distribution of Training Images
20
30
40
50
60
70
35.00%
0.01
0.27
0.38
0.24
0.10
0.00
0.00
30.00%
29.40% 28,
51%
Male
Reference Input
0.00
0.22
0.38
0.28
0.11
0.01
0.00
40.00%
Female CALUUER
25 03.007%
20.007%
18,7875
0.02
40
15.00%
11.57%
0.00
0.15
0.03
Output
10.00%
6,42%
0.00
0.08
0.37
0.22
0.05
0.01
Reference Input
Male Remale
5.00%
0.95%
0.00
0.07
0.20
0.35
0.25
0.1
0.03
0.87
0.13
0.00%
10.00)
[30 [0)
(20.0)
140, 30
180 mg)
pe 70)
(70)0
70
0.00
0.04
0.16
0.26
0.27
0.17
0.10
0.39
0.61
Attribute (Age)
Attribute (Gender)
Figure 4: Facial attribute evaluation in terms of cache side channels. Confusion matrices (with row-wise normalization) compare the age and gender classification results on our recovered images and those obtained from reference inputs. We also report the age and gender distributions in the training set for the reference.
Table 3: Generalizability evaluation.
N)
(1, 100)
(5. 100)
(20, 100)
(1. 500)
(5. 5.000)
(20, 500)
UPNG: SCA model trained with Celend 1ibipeg: SCA model trained with mini-Imagenet Baseline
18.8%
40.6%
66.0%
8.3%
20.7%
38.4%
4.4%
17.1%
39,8%
1.1%
5.0%
15.0%
random guess
1%
5%
20%
0.2%
1%
4%
5.3 GENERALIZABILIT!
This section explores the generalizability of our SCA. We launch attacks toward uPNG to illustrate that our method is independent of specific software implementation or image formats. For uPNG experiments, we evaluate attacks on the Cel ebb dataset using cache side channels. As shown in table Table 3 attack recover discriminable images and largely outperforms the bascline (random guess) in terms of privacy inference. See E the reconstructed images. We also benchmark our attack synthesize arbitrary images without using specific types of images train the model. We instead use a general training dataset, mini- Imag-mnteeet. We use cache side channels to exploit 1 lib jpeg. Table TableBiillsseel iillustrate that considerable privacy is leaked using mini-Imagener the training set, and for all settings, our SCA largely outperforms the baseline.
The recovered images when using mini-Imagenet the training data are visually worse than images recovered using specialized datasets. See Appendix [F for the recovered images. This observation reveals the potential trade-off of our research. Overall, training generative models using a general dataset without the knowledge of images classes seems "unconventional." A generative model is typ- ically trained using dataset of only one class (Guo al. 2019). or it requires image class information to be explicitly provided during both training and generation phases (Brock et al.) 2019). Neverthe- less, we still evaluate our approach using a general dataset to explore the tulf potential of our attack. We attribute the adequate results using general datasets to discriminable features extracted by our trace encoder from images of different classes. See Appendix for further evaluations.
From holistic perspective, the adopted training image sets constitute predictive model of user privacy. While a particular user input is private, we assume that the functionality of victim software (e.g., human face analysis service) is usually known to the public or can be probed prior to attacks.
6 DISCUSSION
This is the first paper that provides a practical solution to reconstruct images from system side chan- nels. Proposing a novel generative model design is not our key focus. Also, despite the encouraging results, the reconstructed images show room for improvement. For instance, not all image colors were well recovered. Our manual inspection shows that compared with libipeg, uPNG does not impose informative side channel dependency on pixel colors (i.e., different colors can likely induce identical side channel logs). Nevertheless, user privacy is presumably leaked as long the image skeleton is recovered. Colors (or other details) can be recovered if the system community discovered more powerful (finer- -rainedd side channels.
8

---

Published as a conference paper at ICLR 2021
7 RELATED WORK
Exploiting System Side Channels. System side channels have been used to exploit various real-life
software systems (Dong al. 2018 Wu et 2012, Hähnel et al. 2017 Xu et al. 2013). The CPU cache is shown to be rich source for SCA attacks on cloud computing environments and web browsers (Hähnel et al. 2017 Oren et al. 2015). 5). In addition to cache line side channels analyzed in this research, other cache storage units, including cache bank and cache set, are also leveraged for SCA attacks (Yarom et al. 2017
1 Liu et al. 2015a). Overall, while most attacks in this field perform dedicated SCA attacks toward specific side channels, approach is general and orthogonal to particular side channels.
Profiling- based SCA. Machine learning techniques have substantially boosted profiling-based SCA by learning from historical data. DNN models have been used to recover secret keys from crypto libraries under different scenarios (Heuser & Zohner 2012 Maghrebi et al.) 2016 Cagli al. 2017 Hettwer et al.) 2018 Kim et 2019 Hettwer et al. 2020). Nevertheless, the success of existing AI- based SCA attacks primarily driven by the model classification capability, e.g., deciding whether a particular bit of AES secret key is 0 or 1. This paper advocates the new focus on reconstructing images with generative models, leveraging another major breakthrough in DNN.
SCA Mitigation. Existing SCA mitigation techniques can be categorized into system-based and software-based approaches. For system-based approaches, previous works have proposed ran- domize the cache storage units or enforce fine-gr fine-grained isolation schemes (Wang & Lee 2006 2008
2007 Liu et al. (016)) Some recent advances propose to leverage new hardware features to mitigate side channel attacks (Gruss et al. 2017). In addition, software-level approaches, including design- ing secret-independent side channel accesses, randomizing memory access patterns (Coppens al.) 2009 Raj et al. 2009 Schwarz et al. 2018). have also been proposed. Compared with system- and hardware-baat baser mitigations, software- approaches usually do not require a customized hard- ware design, and are generally more flexible. Nevertheless, software-based approaches can usually incur extra performance penalty.
8 CONCLUSION
This paper has presented a general and effective SCA framework. The framework trained with side channels to exploit media software like 1 libjpeg and uPNG. Our evaluation shows that recon- structed images manifest close similarity with user inputs, making privacy leakage attacks practical. We also show surprising findings that enabled by our framework, attacks with low-resolution side channels become feasible.
ETHICS STATEMENT
We present a systematic and effective pipeline of recovering private images using system side chan- nels. is generally acknowledged that studying attack schemes helps eliminate false trust on modern computing infrastructures and promote building secure systems (Athalye et al. 2018). While there risk that SCA could become easier using our methods, we believe that our work will also pro- mote rapidly detecting SCA before security breaches. As will be shown in Appendix[] our proposed technique can serve as a "bug detector" isolate certain code blocks in image processing software that induce SCA opportunities. Developers can thus refer to findings to patch their software.
Our efforts could impact the ever-growing CV community in building side channel-free image anal- ysis tools. Despite the algorithm-level efforts to address privacy concerns, e.g., via differential privacy (Dwork et al. 2006), the infrastructure-level vulnerabilities have not yet received enough attention, especially in the real-world scenarios like MLaaS. Our research will serve critical incentive to re-think tradeoffs (e.g., cost vs. security guarantee) currently taken in this field.
10 ACKNOWLEDGEMENTS
We thank the ICLR anonymous reviewers and area chairs for their valuable feedback. Junping Zhang is supported by National Key Research and Development Project 13051301004

---

Published as a conference paper at ICLR 2021
REFERENCES
Face attributes service.
Sttpg:/.hggsss////../..llllllllt 2020.
PAGE MASK.
x86 httb/inccude/aaee wwwpo/udddaa///aaaeeteroee http:///h....looooooooooooaossl/
2020.
Onur Aciicmez and Cetin Kaya Koc. Trace-driven cache attacks on AES. ICICS, 2006.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv: M.Xiv:8802.002220 2018.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2019. URL
https://openrew//
net. htte///oummmmmillxie/.
Eleonora Cagli, Cécile Dumas, and Emmanuel Prouff. Convolutional neural networks with data aug- mentation against jitter-based countermeasures. In International Conference Cryptographic Hardware and Embedded Systems, pp. 45 68. Springer, 2017.
R. C. Chiang, S. Rajasekaran, N. Zhang, and H. H. Huang. Swiper: Exploiting virtual machine vul- nerability in third-party clouds with competition for i/o resources. IEEE Transactions on Parallel and Distributed Systems, 26(6):1732-1733 June 2015. ISSN 1045-9219.
Alex Clark. Pillow. Python image analysis library, 2020.
URL https://pillow
readthedocs htw.aeedddoo...oo///ssataaea stable/
Bart Coppens, Ingrid Verbauwhede, Koen De Bosschere, and Bjorn De Sutter. Practical mitigations for timing-based side-channel attacks on modern x86 processors. In IEEE SP, 2009.
Emily Denton and Rob Fergus. Stochastic video generation with learned prior. In International Conference on Machine Learning, pp. 1174-1183, 2018.
Xiaowan Dong, Zhuojia Shen, John Criswell, Alan L Cox, and Sandhya Dwarkadas. Shielding software from privileged side-channel attacks. 27th USENIX Security Symposium, pp. 1441- 1458, 2018.
Goran Doychev, Dominik Feld, Boris Kopf, Laurent Mauborgne, and Jan Reineke. CacheAudit: A tool for the static analysis of cache side channels. In USENIX Sec., 2013.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. Theory of cryptography conference, pp. 265-284. Springer, 2006.
Daniel Gruss, Julian Lettner, Felix Schuster, Olya Ohrimenko, Istvan Haller, and Manuel Costa. Strong and efficient cache side-channel protection using hardware transactional memory. In USENIX Sec., 2017.
David Gullasch, Endre Bangerter, and Stephan Krenn. Cache maanes-bringing access-based cache attacks on ABS to practice. In Proc. IEEE Symp. Security and Privacy (S&P), pp. 490-505, 2011.
Yong Guo, Qi Chen, Jian Chen, Qingyao Wu, Qinfeng Shi, and Mingkui Tan. Auto-embedding generative adversarial networks for high resolution image synthesis. IEEE Transactions on Mul- limedia, 21(11):726-2237, 2019.
Marcus Hähnel, Weidong Cui, and Marcus Peinado. High-resolution side channels for untrusted operating systems. In 20/7 USENIX Annual Technical Conference, pp. 299-312, 2017.
Y. Han, J. Chan, T. Alpean, and C. Leckie. Using virtual machine allocation policies to defend against co-resident attacks in cloud computing. IEEE Transactions on Dependable and Secure Computing, 14:95-108, Jan 2017.
10

---

Published as a conference paper at ICLR 2021
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Benjamin Hettwer, Stefan Gehrer, and Tim Güneysu. Profiled power analysis attacks using convo- lutional neural networks with domain knowledge. In International Conference on Selected Areas in Cryptography, pp. 479-498. Springer, 2018.
Benjamin Hettwer, Tobias Horn, Stefan Gehrer, and Tim Güneysu. Encoding power traces as images for efficient side-channel analysis. arXiv preprint arXiv:2004. 1015, 2020.
Annelie Heuser and Michael Zohner. Intelligent machine homicide. In International Workshop Constructive Side-Channe Analysis and Secure Design, pp. 249-264. Springer, 2012.
Sanghyun Hong, Michael Davinroy, Yigitcan Kaya, Stuart Nevans Locke, lan Rackow, Kevin Kulda, Dana Dachman-Soled, and Tudor Dumitraş. Security analysis of deep neural networks operating in the presence of cache side-channel attacks. arXiv preprint arXiv: 18/0.03487 2018a.
Sanghyun Hong, Michael Davinroy, Yigitcan Kaya, Dana How to Own the nas your spare time. International Conference on Learning Representations, 2020. URL htt.tttts:/
and Tudor Dumitraş.
ew net. forttfoormm? d= Seees1eerpBFFF
Seunghoon Hong, Dingdong Yang, Jongwook Choi, and Honglak Lee. Inferring semantic layout for hierarchical text-to-image synthesis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7986-7994, 2018b.
Gabriel Hospodar, Benedikt Gierlichs, Elke De Mulder, Ingrid Verbauwhede, and Joos Vandewalle. Machine learning in side-channel analysis: a first study. Journal of Cryptographic Engineering, 1(4):293, 2011.
Intel. Intel software guard extensions programming reference. com/sites sttss/tts////// de waafflllllleeeeee /tt.e///eeeeateeeeee/eeeeeeaeeee
https: htwp:////.e//oooce////////222929
intel.
2014. Order
Number: 329298 00088 October 2014.
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 112--1134, 2017.
Jaehun Kim, Stjepan Picek, Annelie Heuser, Shivam Bhasin, and Alan Hanjalic. Make some noise. unleashing the power of convolutional neural networks for profiled side-channel analysis. IACR Transactions on Cryptographic Hardware and Embedded Systems, pp. 148-179, 2019.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2014.
Ivan Laptev and Tony Lindeberg. Velocity adaptation of space-time interest points. In Proceedings of the 17th International Conference on Puttern Recognition, 2004. ICPR 2004., volume 1, pp- 52-56. IEFE, 2004.
libjpeg.
Main libjpeg- turbo repository,
2020.
URL https: com/
lib jog--turre
Liu, Q Ge, Y. Yarom, Mckeen, C. Rozas, G. Heiser, and R. B. Lee. Catalyst: Defeating last-level cache side channel attacks in cloud computing. In //PCA, 2016.
Fangfei Liu. Yarom, Qian Ge, G. Heiser, and R.B. Lee. Last-level cache side-channel attacks are practical. In 2015 IEEE Symposium on Security and Privacy, pp. 605-622, May 2015a.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 201
Chi-Keum Luk, Robert Cohn, Robert Muth, Harish Patil, Artur Klauser, Geoff Lowney. Steven Wallace, Vijay Janapa Reddi, and Kim Hazelwood. Pin: building eustomized program analysis tools with dynamic instrumentation. In Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation (PIDD) 2005.

---

Published as a conference paper at ICLR 2021
Houssem Maghrebi, Thibault Portigliatti, and Emmanuel Prouff. Breaking cryptographic imple- mentations using deep learning techniques. In International Conference on Security, Privacy, and Applied Cryptography Engineering, pp. 3-26. Springer, 2016.
Sean Middleditch. upng, 2010, URL
Tae-Hyun Oh, Tali Dekel, Changil Kim, Inbar Mosseri, William T Freeman, Michael Rubinstein, and Wojeiech Matusik. Speech2face: Learning the face behind a voice. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 7539-7548, 2019.
Yossef Oren, Vasileios P Kemerlis, Simha Sethumadhavan, and Angelos D Keromytis. The spy in the sandbox: Practical cache attacks in javascript and their implications. In Proceedings of the 22nd ACM SIGSAC Conference on Compuler and Communications Security, pp. 1406-1418, 2015.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In Proceedings of 4th International Conference on Learning Reprexentutions, 2016.
Himanshu Raj, Ripal Nathuji, Abhishek Singh, and Paul England. Resource management for isola- tion enhanced cloud services. In CCSW, 2009.
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In International Conference on Machine Learning, 2016.
Michael Schwarz, Moritz Lipp. Daniel Gruss, Samuel Weiser, Clémentine Maurice, Raphael Spre- itzer, and Stefan Mangard. KeyDrown: Eliminating software-based keystroke timing side-channel attacks. In NDSS, 2018.
Shuai Wang. Pei Wang, Xiao Liu, Danfeng Zhang, and Dinghao Wu. CacheD: Identifying cache- based timing channels in production software. In 26th USENIX Security Symposium, pp. 235-252, 2017.
Shuai Wang, Yuyan Bao, Xiao Liu, Pei Wang, Danfeng Zhang, and Dinghao Wu. Identifying cache- based side channels through secret-augmented abstract interpretation. In 28th USENIX Security Symposium, pp. 657-674, 2019.
Zhenghong Wang and Ruby B. Lee. Covert and side channels due to processor architecture. In ACSAC, 2006.
Zhenghong Wang and Ruby B. Lee. New cache designs for thwarting software cache-based side channel attacks. ISCA, 2007.
Zhenghong Wang and Ruby B Lee. A novel cache architecture with enhanced performance and security. In MICRO, 2008.
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility structural similarity. IEEE transactions on image processing, 13(4):600- 612, 2004.
Yandong Wen, Bhiksha Raj, and Rita Singh. Face reconstruction from voice using generative adver- sarial networks. Advances in Neural Information Processing Systems, pp. 9266-5275, 2019.
Zhenyu Wu, Zhang Xu, and Haining Wang. Whispers in the hyper-space: High-speed covert channel attacks in the cloud, In Presented as part of the 21st USENIX Security Symposium (USENIX Security 12), pp. 159-173, 2012.
Yuanzhong Xu, Weidong Cui, and Mareus Peinado. Controlled-channel attacks: Deterministic side channels for untrusted operating systems. In 2015 IEEE Symposium on Security and Privacy, pp- 640-656. IEEF, 2015.
Yuval Yarom and Katrina Falkner. FLUSH+RELOAD: A high resolution, low noise, L3 cache side-channel attack. In Proceedings of the 23rd USENIX Conference on Security Symposium, pp. 719-732, 2014.
12

---

Published as a conference paper at ICLR 2021
Yuval Yarom, Daniel Genkin, and Nadia Heninger. Cachebleed: timing attack on openssi constant- time Isa. Journal of Cryptographic Engineering, (12),99-112, 2017.
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. LSUN: Construction of large-scale image dataset using deep learning with humans in the loop. arXiv preprint
03365, 2015.
Richard Zhang, Phillip Isola, Alexi A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
13

---

Published as a conference paper at ICLR 2021
A ATTACK SETUP DETAILS
In this section, provide detailed information regarding attack setup, including three employed side channels, the attacked libjpeg and uPNG libraries (libjeeg 2020 Middleditch 2010), and how we log side channel information. Three side channels are taken into account follows:
. Cache Line. Cache line side channel denotes one popular hardware side channel, enabling the exploitation of weal-world crypto, image and text libraries (Hähnel al. 2017] Yarom & Falkner, (014)). The CPU cache, as one key component in modern computer architecture, stores data so that future memory requests of that particular data become much faster. Data are stored in cache block of fixed size, called the cache line. Each memory access made by victim software is projected to a cache line access. In typical cloud platforms, an attacker monitor cache line accesses made by victim software, leading to powerful side channel. For modern Intel architectures, the cache line index of a memory address addr can be computed as addr > 6. Therefore, access of particular cache line can be mapped back to 2 memory addresses.
Page Table. The OS kernel uses the page table track mappings between virtual and phys- ical memory addresses. Every memory access made by the victim software is converted into physical address by querying page table entry. cloud computing platforms, a malicious OS on the host can observe page table accesses made by the victim software to infer its memory access (Xu et al. 2015). Given virtual address addr, we calculate the accessed page table index by masking addr with PAGE.MASK M: addr & (~ M). M is 4095 on modern x86 architectures (pag 2020p.
Data Read/Write Access. Our preliminary study shows surprising results that enabled by powerful generative models, low-resolution side channels of only one-bit data read/write access records be used recover quality images. That is, given memory access made by the victim software, we use one bit to note whether the access is a read or write operation. Such data read/write accesses can be easily observed by monitoring either cache or page table accesses.
SCA attacks using cache lines and page tables are well-known and have enabled real-life exploita- tions in various scenarios. In contrast, to our knowledge, no real-world attacks are designed to exploit read/write patterns. This work shows that quality images can be synthesized by using such low-resolution read/write access side channels.
Consistent with existing SCA of exploiting media software (Xu et al., 2013 Hähnel et al. 2017), we use widely-used JPEG image processing library, libipeg, as the attack target. Attacking 1 lib jpeg which has been exploited in the literatures makes it easier to (conceptually) compare
Preparing Vietim Software
approach with existing works. As mentioned in our paper, we indeed contact authors of both papers to inquire their tools; we didn't 't receive any response by the time of writing. As disclosed in their papers, manual efforts are primarily used recover images. On the other hand, there is no issue for our approach to analyze other image processing libraries as long as different inputs adequately influence side channel logs. To demonstrate the generalizability of our approach, we also attacked another image library, uPNG, which takes images of PNG format as the inputs.
JPEG and PNG are two popular image compression standards. Given an image of JPEG/PNG formats, both image processing libraries reverse the compression step to generate a bitmap image as the prerequisite of many image analysis applications. The decompression process introduces considerable amount of input-dependent side channel accesses for both libraries. We compile both 1ib jpej and uPNG 64-bit Ubuntu 18.04 machine with gec with optimization-level as -00 which disables all optimizations.
We measure the complex of libipeg, by counting the line of code of the attacked 1ib jpeg module. We report the attacked module, conducting JPEG image decompression under various settings, has approximately 46K lines of code. Similarly, the uPNG software has about 1.2K lines of code. In contrast, typically the crypto software attacked by previous profiling-
booseing--bsedd based SCA (Hettwer Yarom al. 2017) are much simpler. For et al. 2020 Gullasch al. 2011 Actiemez & Koc
2006
14

---

Published as a conference paper at ICLR 2021
instance, the x86 implementation of Advanced Encryption Standard (AES) in OpenSSL has about 600 lines of code (excluding global data structures like permutation boxes).
Preparing Side Channel Logs To prepare side channel traces, we use Pin (Luk et al. 2005). runtime monitoring framework developed by Intel to intercept all memory accesses of our test programs when processing an input image i. Every virtual address addr on the logged trace is translated into its corresponding cache line and page table indexes following the aforementioned methods. Similarly, we intercept all memory accesses, and for each memory access, we use one bit to denote whether is data read or write access. All these runtime monitoring tasks can be done by writing two plugins of Pin. We report that processing each image with 1 libipeg can generate trace of 730K 760K side channel records. Processing an image with uPNG can generate trace of about 1.3M side channel records. Recall as introduced in Sec. 3.1 each side channel trace is encoded into a N N x K matrix and then processed by CNNs. into a 512 x 512 x 3 matrix. A uPNG trace is encoded into 512 x 512 x 6 matrix. We do zero padding for matrices. comparison, exploiting crypto libraries (e.g., AES decryption) generates much succinct side channel traces with only few hundred records (Hettwer et al. 2020).
libjpeg trace encoded
Attacking Other Image Processing Software and DNN Models We pick libipeg since this the only media software attacked by existing SCA (Xu et al. 20155 Hähnel al. (017). We also attacked uPNG to demonstrate the generalizability of our approach. Note that libjpeg is commonly used in the image analysis pipeline, e.g., it the prerequisite of the popular Python image processing library - Pillow (Clark 2020).
Also, careful readers may wonder the feasibility of directly exploiting DNN-based image analy- sis tools. However, as pointed out in previous research (Hong et al.) 20188)..... memory access of typical DNN operations like matrix multiplications is not input-dependent. That is, while it has been demonstrated by the same authors that cache side channels is feasible to recover DNN model architectures (Hong et al. 2020), SCA is generally not feasible to recover inputs to DNN models.
B MODEL ARCHITECTURE AND EXPERIMENT SETUP
We now report the architecture and parameters of our framework. The Enerraoe of VAE-LP module is reported in Table [ Encimage and Decamage are reported in Table The generator G and discriminator D our GAN module are listed Table 6 and Table 7 respectively.
We implement our framework in Pytorch (ver. 1.5.0). We use the Adam optimizer (Kingma & Ba. 2014) with learning rate AE-LP = 0.0001 for the VAE-LP module, and learning rate AGAN 0.0002 for the GAN module. We set 81 0.5, and 82 0.999 for both modules. in Lossy AE LP is 0.0001, and 7 in LOSSGAN is 100. Minibatch size 50.
We ran experiments on Intel Xeon CPU E5-2678 with 256 GB of RAM and one Nvidia GeForce RTX 2080 GPU. The training is completed at 200 iterations (100 iterations for the VAE-LP module, and 100 iterations for the GAN module).
15

---

Published as a conference paper at ICLR 2021
Table 4: The image encoder Emos and image decoder DeCimage architectures.
Image Encoder
Image Decoder
Layer
Act.
Output shape
Layer
Act.
Oulput shape
Input
3 x 128 x 128
Input
128 x x
Conv 4 X 4/2,
(ReLU(0.2)
128 x 64 64
Deconv 4 x 4/100
BN + ReLU
128 x 4 x 4
ResBlock (He al. 2016
128 64
64
ResBlock
128 x 4 4
Conv 4 x /
BN +.ReI.U(00
128 32
Deconv 4 4/2,1
BN ReL.U
128
8
ResBlock
128 x 32
32
ResBlock
128 x
8
Conv x 4/2,1
BN + (.RCLU(0
128 x 16
16
Deconv 4 x 4/2,1
BN + ReLU
128 16 x 16
ResBlock
128 X 16 16
ResBlock
128
16 x 16
Conv x 4/2,1
BN LReLU(0.22
128
Deconv 4 4/2,1
BN ReLU
128
32
ResBlock
ResBlock
128
32
Conv x 4/2,1
BN + +RReLU(0.2)
128 X 4 4
Deconv 4 x 4/2,1
BN + ReLU
128 x 64 x 64
ResBlock
128 X 4 X 4
ResBlock
128 64 x 64
Conv x 4/1,0
LReLU(0.2)
128 1 1
Deconv 4 4/2,1
Tanh
3 3 128 128
Output
128
Output
3 128 128
Table 5: The trace encoder Enctrace architecture.
Trace Encoder
Layer
Act.
Output shape
Input
3 512 x 512
Normalize
3 x 512 512
Conv 4 4/2,1
BN (ReL.
64 256 256
Conv 4 4/2.1
BN LReLU(0.2)
64 x
Conv 4 4/221
BN LReLU(0.2)
64 x 64 x 64
Conv 4 4/221
BN LReLU(0.22)
128 32 x 32
Conv 4 4/2,1
BN LReLU(0.2)
256 16 x 16
Conv 4/2,1
BN LReLU(00.2
512 x x
Conv 4 4/2.1
BN + LReeU((0.2)
512 X 4
Conv 1 x 4/1.0
BN + Tanh
128 x x 1
Output
128
Table 6: The generator G architecture.
Encoder
Decoder
Layer
Act.
Output shape
Layer
Act.
Output shape
Input
3 x 128 x 128
Input
1024 x x
Conv 4
4/2,1
128 x 64 64
Deconv 4 x 4/2,1
BN + ReLU
1024 x x 2
Conv 4
4/2,1
BN (Re
256 32 32
Deconv 4 4/2,1
BN ReLU
1024 x 4
Conv 4 4/2,1
+ (0.LU(0.22)
512 x
16
Deconv x 4/2,1
+ ReLU
1024 x
Conv 4 x 4/2.1
BN + LReLU(0..)
1024 x 8 x 8
Deconv 4 x 4/2,1
BN + ReLU
512 x 16 x 16
Conv 4 x 4/2,1
BN + LReLU(0022
1024 X 4 x
Deconv 4 x 4/2,1
BN + ReLU
256 x 32 x 32
Conv 4 4/2,1
BN + LReLU(00.)
1024 x x 2
Deconv 4 x 4/2,1
BN + ReLU
128 64
64
Conv 4 4/2,1
LReLU(0.2)
1024
Deconv 4/2,1
Tanh
3 128 x 128
Output
1024 X
Output
x 128 X 128
Table 7: The discriminator D architecture. Discriminator
Layer
Act.
Output shape
Input
3 128 128
Conv 4 x 4/2,1
BN + LRcLU(0.2)
128 x 64 x 64
Conv 4 x 4/2,1
BN LReLU((..2)
256 32 x 32
Conv 4/2,1
BN LRcLU(0.2)
512 16
Conv x 4/221
BN (ReLU((.2)
1024 x
Conv x 4/2,1
BN + LReLU((0.2)
1024 x x
Conv 1 x 4/1.0
Sigmoid
1 X
Output
1
16

---

Published as a conference paper at ICLR 2021
C SAMPLE OUTPUTS WHEN ATTACKING LIBJPEG
This section provides more images generated by our framework when attacking libjpeg. Overall, we interpret the results as promising and highly consistent across all three different datasets. As dis- cussed in Sec. 5.1 the reconstructed and the corresponding reference images show highly consistent identities, such as gender, face orientation, human gesture, and hair style.
VAE
VAE GAN
VAE
VAELP GAN
VPE-LP
VAE GAN RoNeennc
Read
Table
CPU CudseLine
Figure 5: Evaluation results of CelebA dataset in terms of three different side channels. The reference images (i.e., private inputs) are presented on the right most column for the reference.
VAE-LP
VAE-I & GAN
VAELPP
VAE-L GAN
VAE-LP
VAE-LP RAN leput
Read/WWitc Auceas
08 Page Table
CPU Cne
Figure 6: Evaluation results Ce Lebb dataset in terms of three different side channels. The reference images (i.e., private user inputs) are presented on the right most column for the reference.
17

---

Published as a conference paper at ICLR 2021
VAE-LP
VAE-LP GAN
VAE-LP
VAE-LP GAN
VAB-LP
VAE-LP GAN Referenee Input
Puge
Cuche
Figure 7: Evaluation results of dataset in terms three different channels. The reference images (i.e., private user imputs) are presented on the right most column for the reference.
VAE-LP
VAE-LP GAN
VAE-LP
VAE-LP GAN
VAE-LP
VAE-LP GAN Reference Input
Read/Witt
Page Table
Cuche
Figure 8: Evaluation results of dataset in terms different channels. The reference images (i.e., private user inputs) are presented on the right most column for the reference.
18

---

Published as a conference paper at ICLR 2021
VAE-LP
VAE-LP & GAN
VAE-LP
VAE-LP GAN
VAB-LP
VAE-LP GAN Referenee Input
Read//Wiite
Puge
Cuche
Figure 9: Evaluation results LSUN dataset in terms different side channels. reference images (i.e., private user inputs) are presented on the right most column for the reference.
VAE-LP
VAE-LP GAN
VAE-LP
VAE-LP GAN
VAE-LP
VAE-LP GAN Reference Input
Read/Witt
Page Table
Cuche
Figure 10: Evaluation results LSUN dataset in terms three different channels. The reference images (i.e., private user inputs) are presented on the right most column for the reference.
19

---

Published as a conference paper at ICLR 2021
D QUANTITATIVE EVALUATION
Besides the quantitative evaluation of the discriminability of reconstructed images reported in the paper, we also analyze images reconstructed by using only the VAE-LP module and presented the results in Table 8 Accordingly, we give the quantitative data that has been already reported in our paper for cross comparison in Table
Table 8: Discriminability evaluation results outputs using VAE-LP (higher is better). We also present the baseline, random guess (RG), for comparison.
k N
100
k = 1 N
500
CelebA
KTH
LSUN
RG
CelebA
KTH
LSUN
RG
Cache Line Page Table
49.98%
30.28%
10.36%
1%
31.18%
19.82%
3.30%
0.2%
14.66%
29.00%
1.94%
1%
5.98%
18.20%
0.50%
0.2%
Read/Write
17.52%
24.46%
1.98%
1%
7.86%
14.92%
0.40%
0.2%
5 N
100
k = 5 N
500
Ce Cele
KTH
LSUN
RG
Ce lebA
KTH
LSUN
RG
Cache Line
78.00%
54.42%
29.74%
5%
56.06%
32.86%
11.50%
1%
Page Table
35.38%
52.70%
7.92%
5%
16.92%
31.54%
1.92%
1%
Read/Write Write
39.16%
47.06%
7.08%
5%
19.68%
26.02%
1.86%
1%
k 20 N
100
k 20 N
500
Ce CelebA
KTH
LSUN
RG
CelebA
KTH
LSUN
RG
Cache Line
91.98%
79.169 16%
59.66%
20%
75.98%
51.20%
26.72%
Page Table
61.94%
76.88%
24.90%
20%
32.96%
50.32%
6.48%
4%
Read/Wrice
65.46%
72.92%
24.56%
20%
36.90%
43.78%
6.30%
4%
Table 9: Discriminability evaluation results the outputs using VAE-LP & GAN modules (higher better). We have already reported these results in Table
1 N
100
== 1
500
CelebA
KTH
LSUN
RG
CelebA
KTH
LSUN
RG
Cache Line Page Table
47.28%
38.22%
10.00%
1%
28.68%
24.50%
3.04%
0.2%
16%
35,86%
2.26%
1%
6.56%
22.58%
0.46%
0.2%
Read/Write
19.58%
33.44%
2.24%
1%
8.42%
20.64%
0.50%
0.2%
5
100
5
500
CelebA
КTH
LSUN
RG
Ce CelebA
KTH
LSUN
RG
Cache Line
74.98%
63.56%
28.74%
5%
52.22%
41.24%
10.90%
1 1%
Page Table
37.96%
61.28%
8.62%
5%
17.90%
38.80%
2.42%
1 %
Read/Write
42.44%
58.30%
8.28%
5%
21.18%
36.22%
2.54%
1
k 20 N
100
20 N
500
CelebA
KTH
LSUN
RG
CelebA
KTH
LSUN
RG
Cache Line
90.70%
83.72%
58.88%
20%
72.98%
61.08%
25.92%
4%
Page Table
65.14%
82.22%
26.42%
20%
35.22%
58.90%
7.30%
4%
Read/Write
67.82%
80.14%
26.64%
20%
39.84%
55.74%
7.08%
4%
Comparing results reported in Table Table we observed that by using VAE-LP & GAN mod- ules together, the KTH human gesture dataset has substantial improvement in terms of accuracy.
The average accuracy the KTH dataset 41.9% in Table X while the average accuracy of the
dataset is increased to 49.8% in 9 This observation is consistent with our findings in Sec. 5.2 and some cases demonstrated in Fig. and Fig. g Recall we observed an obscure "black bar" in КTH images reconstructed by only using VAE-LP module, while human gesture can be clearly identified by enhancing the "black bar" with the GAN module. We also observed improved aceu- racy for the Ce CelebA (41.0% to 41.4%) and LSUN datasets (12.6% to 12.9%) when supplementing VAE-LP with GAN. Overall, we interpret that better discriminability can be achieved when using GAN, implying higher success rate for attackers de-anonymize user identity and privacy.
20

---

Published as a conference paper at ICLR 2021
We also present fine- finingr facial attribute comparison between the reconstructed images and refer- ence inputs. In Sec. 5.2 we have reported gender and age confusion matrices evaluation using cache side channels (also presented in Fig. IT for the reference and cross comparison), we also report other settings in Fig. 12 Fig. 13 and Fig. 14
To quantitatively evaluate the confusion matrices, we mea- sure and report the weighted-averaage Fl score in Table TO Besides one with notably increased Fl score (the gender matrix using page table), VAE-LP and VAE-LP & GAN have comparable weighted-average Fl scores.
Output
Gender Distribution Training Images
Age Distribation of Training Images
20
30
40
50
60
70
0.01
0.27
0.38
0.24
0.10
0.00
1.000
20
Male
0.00
0.22
0.38
0.28
0.11
0.01
0.00
30
0.00
0.17
0.38
0.32
0.11
0.02
0.00
20.00%
18.78%
15.00%
Reforeage
0.00
0.13
0.33
0.36
0.15
0.03
0.00
13.57%
Outpul
10.00%
0.05
Reforence Ingut
Mals Faanle
0.87
0.13
5.00%
0.95%
0,00%
(020))
(3030)
[30,0]
[40,10)
[90,0)
(10,20)
mo
0.00
0.27
0.17
0.10
Attribute
Attribure (Gender)
Age
Figure 11: Facial attribute evaluation in terms of cache side channels. Images are generated using VAE-LP & GAN modules. Confusion matrices (with row-wise normalization) compare the age and gender classification results our recovered images and those obtained from reference inputs. We also report age and gender distribution dataset for comparison. We have already reported these results in Fig.
Output
20
40
50
70
0.01
0.50
0.34
0.12
0.03
0.00
0.00
20
0.00
0.41
0.43
0.13
0.02
0.00
0.00
Reference Input
0.00
0.37
0.45
0.14
0.03
0.00
0.00
40
0.00
0.32
0.43
0.20
0.04
0.00
0.00
Output
50
Female
Male
0.44
0.23
Reference Input
Male Female
60
0.88
0.12
0.00
0.19
0.38
0.28
0.12
0.03
0.00
70
0.00
0.15
0.31
0.30
0.16
0.07
0.00
0.40
0.60
Attribute (Age)
Attributo (Gender)
Figure 12: Facial attribute evaluation in terms cache side channels, Images are generated using the VAE-LP module. Confusion matrices (with row-wise normalization) compare the age and gender classification results on our recovered images and those obtained from reference inputs. The age and gender distributions in the datasct have been given in Fig. 11
21

---

Published as a conference paper at ICLR 2021
Output
Outpur Male
Outpur
20
60
20
30
40
50
601
Reference Input
Male Female
0.00
0.35
0.44
0.17
0.02
0.01
0.00
0.87
0.13
0.03
0.36
0.26
0.19
0.11
0.04
0.00
20
20
0.48
0.01
0.36
0.28
0.19
0.11
0.04
0.01
30
0.53
30
Reforence Iapul
0.49
0.20
Aftribute (Gennder
0.01
0.33
0.29
0.20
0.12
0.04
0.01
40
40
0.24
0.47
0.24
0.04
0.00
0.00
Page
VAE-1.P
0.28
0.22
50
0.00
0.23
0.44
0.28
0.04
0.01
0.00
Output
0.22
0.24
0.23
0.02
60
Fenude
60
0.00
0.20
0.39
0.31
0.10
0.01
0.00
Reforence liput
Female
0.01
0.19
0.22
0.22
0.20
0.11
0.04
70
70
0.00
0.18
0.20
0.24
0.19
0.14
0.06
Male
0.46
0.54
Altribulc (Age)
Altribule (Ago)
Attribute (Gender)
Page Table VAE-LP
Page Table VAE-LP
OS Page Table VAE-LP & GAN
Figure 13: Facial attribute evaluation of page table side channels. We present evaluation results by using images reconstructed using only VAE-LP module, and images reconstructed using VAE-LP & GAN. Confusion matrices (with row-wise normalization) compare the age and gender classification results on our recovered images and those obtained from reference inputs. The age and gender distribution in the dataset have been given in Fig. IT
Ourput
70
Fennale
Male
50
60
Referance ligut
Female
0.87
0.13
20
0.00
0.30
0.49
0.18
0.03
0.00
0.00
20
Male
0.49
0.51
0.00
0.28
0.69
0.20
0.03
0.00
0.00
Attribute (Gender)
Reference Input
0.01
0.35
0.31
0.30
0.10
0.02
0.01
40
Reference
0.00
0.26
0.48
0.21
0.04
0.00
0.00
Read Write Access VAE-LP
0.01
0.31
0.28
0.23
0.13
0.04
0.01
50
50
Output
0.01
0.26
0.26
0.24
0.15
0.06
0.02
Feraale
Mole
0.00
0.18
0.01
0.00
Reference Input
Male Fomale
0.83
0.17
70
0.00
0.13
0.39
0.33
0.13
0.02
0.00
0.00
0.22
0.20
0.19
0.12
0.9
0.45
0.55
Altribute (Age)
Autribute (Age)
Attributc (Gender)
Read/ Access VAE-LP
Access VAE-LP & GAN
Read Access
GAN
Facial altribute evaluation evaluation results by using images reconstructed using only VAE-LP module, and images reconstructed using VAE-LP & GAN. Confusion matrices (with row-wise normalization) compare the age and gender classification results on recovered images and those obtained from reference inputs. The age and gender distribution in the dataset have been given in Fig.
Figure 14:
of memory read/write access pattern side channels. We present
Table 10: Weighted-average F1-score (higher is better).
Age
Gender
VAE-LP
VAB LP & GAN
VAB- LP
VAE LP
GAN
CPU Cache Line
0.27
0.27
OS Page Table
0.25
0.26
0.68
0.79
Memory Read/Write
0.25
0.27
0.71
0.71
22

---

Published as a conference paper at ICLR 2021
E SAMPLE OUTPUTS WHEN ATTACKING uPNG
This section provides images generated by our framework when attacking uPNG. While the side channel traces induced by uPNG generally less informative than libjpep (as shown in Table and discussed in Sec. [ still observed high visually consistent identities between the recon- structed images and their reference inputs, including gender, face orientation, hair style, hair length, whether wearing pair of glasses and many other factors.
VAF-EP
VAE GAN
Refirnne Iipux
Figure 15: Evaluation results Celebå dataset terms cache channels.
VAE-LP & GAN
Refirena Iepit
Figure 16: Evaluation results of Celeba ebb dataset in terms of the cache side channels.
VAE
VAF-LP&
Reformee Iepul
Figure 17: Evaluation results CelebA dataset terms of the cache side channels.
23

---

Published as a conference paper at ICLR 2021
F SAMPLE OUTPUTS WHEN TRAINING WITH MINI-IMAGENET
To measure our attack reconstructing arbitrary images without knowing the type of images being processed, Sec. 5.3 reports model training and attack performance using a general dataset, mini- Imagenet. We report that the mini-Imagenet dataset has total 60K images of 100 classes, and we divide each class (with 600 images) into 480 training images and 120 testing images. As result, we have in total 48K images for training and take the other 12K images for testing. While training generative models with general dataset is not the common practice unless image class information explicitly provided (Brock et al., 2019), Table still reports highly encouraging results of the discriminability analysis by largely outperforming the baseline random guess. In this section, provide sample images generated at this step.
The synthesized images from the mini-Imagened dataset is visually much worse that images synthe- sized from specific datasets (e.g., Ce ebA in Fig. 5). Nevertheless, by comparing the synthesized images and their corresponding references (i.e., user inputs), we interpret that images still exhibit high discriminability, in the sense that many visually consistent image skeletons and colors are re- covered, indicating adequate leakage of user privacy.
VAE-LPA GAN
VAF-LP
Refisens Ieput
Figure 18: Evaluation results mini- Imagenet dataset when attacking libipeg with cache channels.
VIE GAN
Refires lepat
Figure 19: Evaluation results mini- mini-Immaeent datasct when attacking 1ib jpeg with cache side channels.
24

---

Published as a conference paper at ICLR 2021
VAELLI& GAN
VAB-IP
Reference leput
Figure 20: Evaluation results of mini-Imagenet dataset when attacking libjpeg with cache side channels.
G CLASSIFYING OUTPUTS OF THE TRACE ENCODER
By training our framework with mini-Imagenet and assessing the discriminability, Sec. 5.3 has demonstrated that our attack can largely outperform the baseline even with no prior knowledge on the class information of user private images. We attribute the promising evaluation results to the discriminable features successfully extracted by our trace encoder (trained with mini- mini-Imagenet; see Appendix F). This section presents empirical results by assessing to what extent the latent represen- tations derived from images of two randomly selected classes are distinguishable.
To this end, we build a binary classifier, by piggybacking our trace encoder with a fully-connected (FC) layer and using Sigmoid as the activation function. As mentioned in Appendix [F] the mini- Imagenet dataset has in total 60K images of 100 classes, and we divide each class into 480 images for training and 120 images for testing. At this step, we randomly select two classes of images, and use their training sets (in total 960 images) to train the proposed binary classifier. We then use their testing sets, including total 240 images, to measure the binary classification accuracy. It is worth mentioning that we only train the classifier for one epoch to highlight that the latent representations extracted by the trace encoder already exhibit high quality and discriminability. We only tune the parameters of FC layer but preserve the parameters of our trace encoder.
We iterate this process for 100 times. Fig. reg.ortss the classification accuracy across all 100 binary classification tasks. While the baseline accuracy for our binary classification task is 50% most tasks exhibit much higher classification accuracy. We report the average accuracy is 81.6% and 32 cases exhibit a classification accuracy above 90%.
Random Guoss
0.9
0.7
Acorncy
0.6
0.5
0.4
0.3
Binary Classifiers
Figure 21: Classification accuracy across 100 binary classifiers.
25

---

Published as a conference paper at ICLR 2021
H ROBUSTNESS TO NOISE
This section provides experiences on the robustness of our framework by inserting noise into the side channel traces. To this end, evaluated the following three settings:
Gaussian noise insertion: For each side channel data point input on the side channel trace, inpul = z x n (1 -x) imput, where æ E [0.1, 0.2, 0.5]. and n denotes randomly
generated noise following the Gaussian Distribution.
Zero replacement: Randomly set x% of the data points on the side channel trace to 0,
where [10, 20,50)
Round shifting: Round shifting the side channel trace for steps, where E [1. 10, 100].
Reference Imput.
x=011
x
x 0.5
Figure 22: Evaluation results of CelebA dataset in terms of cache side channels and three different noise setting. We also present the synthesized images with no noise for the reference. User input images are presented the first row for the reference.
Refermace Ingxt.
X x=0
x =
20
=00
Figure 23: Evaluation results of CelebA dataset in terms of cache side channels and three different random zero replacement setting. also present the synthesized images no zero replacement (r 0) reference. User input images are presented on the first row for the reference.
26

---

Published as a conference paper at ICLR 2021
Table 11: Discriminahility evaluation by adding Gaussian noise in the side channel
Configuration
0.1
0.2
==
0.5
k= N =
=00
49.98%
48.32%
39.14%
3.66%
78.00%
76.28%
67.08%
19.62%
N 100
91.98%
90.56%
86.22%
45.20%
Table 12: Discriminability evaluation by randomly replacing x% side channel data points with zero.
Configuration
==0
==
Jr 20
=
50
T=
45.02%
38.50%
14.40%
À = 100
78.00%
73.30%
65.52%
35.50%
k= 20 N = 100
91.98%
89.42%
85.86%
64.14%%
Reference Input
x 0
x=1
==10
=000
Figure 24: Evaluation results of Ce ebA dataset in cache side channels and three different shifting no shifting 0) for the reference. User input images
setting. We also present the synthesized images are presented on the first row for the reference.
We present the corresponding qualitative evaluation results in Fig. Fig. and Fig. 24 tively. Accordingly, present the quantitative results in Table II Table 12 and Table B
respec-
Overall, despite the challenging settings, we still observed considerable visually consistent features (e.g., face orientation, hair style, gender) between the reconstructed images and their reference in- puts. Fig. 24 shows that round shifting seems to impose relatively low impact on the reconstructed images (e.g., comparing
with 100). In contrast, Gaussian noise to each side channel data point, causes observable effect on the constructed images (e.g., comparing with 0.5).
challenging setting, adding
27

---

Published as a conference paper at ICLR 2021
Table 13: Discriminability evaluation by round shifting the side channel trace.
Configuration
10
100
k= N = 100
49.98%
49.56%
49.94%
47.48%
5 78.00%
75.30% 77.42%
77.00%
20 N 100
91.98%
91.44%
91.52%
90.66%
I ABLATION EXPERIMENTS
This section provides more ablation experiments. We aim to demonstrate the necessity of image encoder and learned prior. To this end, we launch experiences to synthesize images without using the image encoder (see the 4th row of Fig. 25). and also synthesize images with fixed Gaussian prior (the 3rd row of Fig- p5p. It is easy to see that the reconstructed images manifest much lower quality compared with images synthesized by our framework (the 2nd row of Fig. 25 In particu- lar, images synthesized without using the image encoder are seen to contain grids (the last row of Fig. 25), It is also observed that when feeding the decoder with fixed gaussian prior, the synthe- sized images are low quality as well. The outputs become not recognizable since the fixed prior has primarily no information of side channel traces. This also indicates that model is image generator, and the trace encoder plays a vital role in the pipeline. Overall, we interpret these ablation evaluations highlight the importance of the image encoder and learned prior in SCA. Our designed framework, by incorporating the side channel trace encoder, can effectively extract latent representations from the logged side channel data points. Simultaneously, by integrating cor- responding reference images during training, we provide guideline to help the image decoder to generate quality images.
simple
Reference Input
Leamed Prior
Image Encoder
Fixed Prior
Image Encoder
Figure 25: Evaluation results of Celebà dataset in terms of cache side channels and different settings. User input images are presented on the first row for the reference. Images synthesized by our presented framework are presented on the second row.
In addition, we further conduct another ablation experiments regarding image level metrics. To do so, we use LPIPS (Zhang et al., 2018), image-level perceptual loss, to calculate the similarity of the reconstructed image and ground truth image.
The results are given in Table 14 Compared with our results reported in Sec. the accuracy of GAN output is reduced by around 10% and reduced even more output of VAE-LP. Nevertheless, the results are reasonable since the model is not train using this perceptual loss. Overall, we interpret the evaluation results show that from the perspective of "human adversaries". the GAN module is indeed necessary.
28

---

Published as a conference paper at ICLR 2021
Table 14: Ablation experiments with LPIPS.
Model
K I.
100
K
100
K 20, N 100
VAE-LP
23.02%
47.92%
72.40%
VAE-LP & GAN
39.64%
63.12%
81.42%
forward propagation
backward propagation
Ux35...aa2
T
Ux35...e61
0x35... 6ce
folded
0x55.. d4c
trace
0x35... d.0
VAE-LP
0x35... an2
T
1x335 462
0x55.,
unfolded
0x55.. d50
Figure 26: Calculating gradient using backward propagation.
MAPPING SIDE CHANNEL TRACES BACK TO INFORMATIVE FUNCTIONS
In this section, we explore side channel traces and aim to answer the question "what information in the side channel is used for image reconstruction?" To this end, we explore which data points on the side channel trace affects most the output by calculating the gradient. Due the limited time, we tested cache side channel traces logged for the 1 lib jipe software.
Given an image which not in training data, we first collect corresponding cache side channel trace T, when being processed by 1 libipeg. VAE-LP module will then take T, as the input and reconstruct étrace. As shown in Fig. 26 we then calculate the loss étrace and i, and further perform backward propagation calculate the gradient up to T, namely gr, Since gr, has the same size of Tü we can pinpoint which part of T; contributes most to itrace by localizing which part of gr, has higher value. More specifically, threshold T (7 is set as 0.8 in our current study). Overall, we report that from the employed cache side channel trace with 754139 data points, successfully pinpoint set of 31 data points that primarily contribute to the private image reconstruction (see Fig. 28 and Fig. 29
normalize 9r, to [0, 1] and only keep values greater than
Since each side channel data point is converted from memory address (see Table D. our retained "informative" side channel data points can thus be mapped back to certain functions in 1 lib libipeg. That indeed use informative side channel records isolate functions in lib Libipeg that poten- tially leak privacy. Fig. 27 depicts how this backward mapping and isolation are conducted. For in- stance, given side channel record 0x55dbal628861 marked informative and notably contributes to the gradient, we use the address of the corresponding memory access instruction, 0x7f38daafd6b5, oat.. That is, certain input-dependent memory access in js siid_convs amp-float induces this cache line access and eventually contributes the recon- struction of the user private input.
to isolate function js mmdd_cdovvaaeoommmmeomm
Fig. 28 reports part of the logged side channel trace and marks several side channel data points in red which largely affects the gradient. We show their corresponding functions in 1ib ipeg in Fig. 29 Overall, report that this evaluation successfully pinpoints multiple critical functions in the iib jpeg software that has ieput-dependent cache accesses. In particular, we note that this eval- uation helps "re-discover" some functions that have been reported by previous research (mostly
29

---

Published as a conference paper at ICLR 2021
Instruction address
Accessed memory address
0x7f...b0 <jsimd_conve Sesiimd_oonvsap__ff___ float>:
Ox 13f
0x55..
0x35. 882
0x35 082
/*function body*/
Ox 6b5
0x55...c62
0x53 oce
0x7f. 022
0x55...
0x7f...6bd: 00 00 00
T
0x53. d50
0x7f. 6a8
0x55...
0x7f..6a0 <jsimd conadsmpp_/samp>
0x
0x55...d50
/*function body*/
0x7f..6ad: 00 00 00
Figure 27: Mapping "informative" side channel data points back to functions Libjpeg.
with manual effort) as vulnerable to SCA: e.g., functions write. ppm, put_rgb and output.ppm which dump the decompressed raw image the disk.
More importantly, this evaluation helps to pinpoint new functions that contribute to the private im- age reconstruction (and hence become vulnerabilities SCA), such as jsimd.can_ffctt_u slow, jsimd can fdntfdot.... ifast, isimd. convsamp and js jsimd convsamp float. These func- tions primarily conduct image discrete cosine transformation (DCT) and decompression, We inter- pret this as a highly encouraging finding, in particular:
As reviewed in Sec. 2 previous research uses manual effort (Xu et al. 2015 Hähnel al. 2017) or formal methods (Doychev et al. 2013; Wang et al. 2017)) to pinpoint program components that depend inputs, which program-speciin and error- error-prone prone with low scalability.
This research and our study in this section actually reveals a procedure where we leverage gradient to directly highlight which part of the logged side channel trace contributes to the synthesis of outputs. Then, we map the highlighted trace back to where they are derived from in the victim software to isolate vulnerable components (i.e., bug detection tool).
We view it a promising finding: approach depicted in this section is general and be launched fully automatically without requiring manual efforts or formal methods which are usually scalable. As shown in this section, our tentative study not only re-discovers vulnerabilities that were found by previous research, but helps to identify, to our best knowledge, unknown program components that are vulnerable SCA. Looking ahead, we would like explore this direction as follow-up of the present work.
30

---

Published as a conference paper at ICLR 2021
0455/8a11::6044
065ddba1688cc
0x550baa1228ac
0r711412241118
152880
1:550ba1623065
0:550ta1623664
0656
065dhx1667119
6v55cha161717e
0e55dha16796ek
0155cha16200d
065dha
0G3/bb162830c
0x55eba16253c
065ddb316287b
0c5dbe15bbe52
Dc5ddba161ee25
De55eba1620bdc
015500162:097
0c5dbs1617771
7r5sdba161/175
0665
045:db33029/b0
0459eba1014999
0655dba1628occ
0x55dba1628acc
Dc55db31628726
0:55/ba1529:50
0s5tdba1625ec8
0s55oba162:618
0/55dba152ba33
0655tba1677176
0r55cha161717d
15258 0e5tth216hh47
1e55dbx1629666
0655
667788
063d6016/800c
0c3ddb31/28/1e
061dbe166bes1
065idba161e22
1653cha1623d1c
16796
063dfo1667172
0105
De5ddh166200f
0155cha662556
16556
065db116/b928
0c5db31628ccc
0x55cba1628cc4
0c55dbs1628727
06558b0122680
0s55dba16256b:
0x55eba1620e22
0155ebo1629600
0155eba1629ef5
0077386x856688
0c55dbou6677755
0r55dbaa661717
0c5dbaa162c466
1555/ba15the53
0e55dba1611e22
16556ba1629918
0655
1655
0.55dba1516:20
0c 6780
0c5ddb1622873
De53dba161e24
0659dh16629d0
1675d
0155
065dbr16/88s
065dbe16:7173
d24111
1616635
0s55dba
1623669
1155
1655
063/63162880c
0x7ff61d24122
063db33628728
Dx50dba161ef2e
0453eba3620ee1
1655
1655eb
065db=1016612
06548b16677744
0:771614341184
5655bb1662c45
05554a1556995
0x55dba3622578
0x55dba1629018
0655eb0162000
0155dba1522884
0x7rd16243100
0x7ffe1d:41110
065ddb31628727
0155/ba15288500
0x55dba1629988
0x55dba1628643
0155ebo15200oc
0655dbo162:66de
0x7738648ee00
0,201812441110
027
0s55aba
1528725
165561
16550
0655/8a1618018
0555
0655dba1516:04
0offdhd241928
De5idha16lefdd
0655cha16000e
16530h11600660
06536a116766644
065/b111//911
007f010241884
0x7ffc16241190
0r5ddb1662o44
06550
Dx55dba1622004
01550ba1618:999
06550b01616208
0655dbo1628882
0677386355eb44
00ffs16263f10
0s55dbalo286s
0x55eba102:619
0155ebo102861c
@155eb11G29ef
0,771112441888
07716614241944
0x711d1224110
1555
0e55dbal61oof2
0:55dba1616555
0655dbs16285e4
0655
0.55dba162b.555
022
0555d be55
1655dba161e42c
055ddww668811
0655
0x7ff112241120
065861152004
DxSSdba1618b6c
06556b16229188
0665db1106614
0c/961d2411110
065db
Dx50dba162clec
0453dba1620cls
1625e
065d611922940
0777116263194
01558b155bbe5
0r55dba1615a2:
0x550ba16297c8
06556b16628669
6655
0x773B6s856ccc
065dh416688733
0555
1e55dba1629670
1155dba1629668
0655
0655
0.55dha1516:22
027701/241120
07771c1/241113
065ddb2662oood
0x55dba
1516336
165568
1:$56ba1625702
065588
1635
0665db11211997
065dhe1
0a55cha6678ach
0c5ddh11678778
167ecb6
0eSidha161ef12
0s55cha16200
00ff016243180
0455dba16171b8
0d5db:162ooob
0:50be1510986
Dx5odba16257c0
04550ba66290c
0655db116206e
0665db-1616300
8
0s55dba1628ace
0<5dhha122830
0:558ba16288864
0:55dba1625uB8
0:55tha1628840
0655
16266f
0655dba162la54
0/77/162241118
0455dbba6617ibr
ae55dba162cocs
0x5586u1628733
0e55dba16228755
0+55dba1620030
0155chs1616019
1655db
0x77516243100
065ddbe1 6
0r55cha1678ara
1155dba161e431
16203
0655a8a167:600
05558ba1698588
063/b11611100
De5idba1611bb
0x/f611224110
0658b1162978
0x50dba16290d
0455eha1015anf
0655eb11619999
06556b11625004
065d6b1611888
0655db1628acb
0s55dba1623ard
06711112241128
0655841529:88
Dx55dba16298t
0x556ba162:62f
1655d
0y7m410241100
065db316:71b7
0s55eba1617ibe
0677161/222004
06558b11629715
0v55dbalG1ef33
Dx55ebalc1a8f
0155bb1662863
0155eba1629700
0655dbo152ape0
678ddio11688d0
0455dba16288c0
0x71ld1:221110
0:55mai15hha50
0e55dha161o/30
0s55dha1623678
6655d8w16200
0655
0x77dh6244100
061dba16671b1
0rsdba161/1ba
1676861679064
De50dba101dbod
0653dba622572f
165dbr1
0655cba668aach
0c55dha1678806
1698889
BeSSdha162eSed
0s55cha1620eeh
1655
065dba169w67s
063dbo16171b6
Ds53eba6671bd
0cSddb161bb41
b65dbe15bbebb
Dx5odba1611e2
04530ba1629700
06556b166866
0.55/ba1628ac
0455dba1628Buc
br55dba16250:c
0:558b155bbe88
0e55dba162886:
0s55dba1629665
0:550b116200
06556b
0x7Md1624am0
0x55dba1617tb
0e55dhs16776el
155668
8e5Sdba1628789
06550
06550ba1600eff
6784c
1655th2661be43
0s55dib
153ccb7
1r55dba161w436
16556h
0655
007Rd16244140
063db11621186
Desodba661/bbc
0cbdbs1111b60
De5ddba1629/b
0453eha1620687
09536b10110600
0offidid2e1500
0655bb162880c
0x7ffc1d241110
0:5db:1616031
06558b1558864
Dx55dba1625600
D455eba1620644
0655db16620de
0455dba162:660
067R610244150
065db316171b3
0x2ffc1d241f10
0c5dbb1622001
0155dba122974b
0s5odbalo25717
0s5sebalo1aaf2
0155eb11610010
0655eba1616120
Dxffidic241ee
0x55dbb11828oo
6771161d541128
0e5dbba1616911
0555dba15tba5e
0e55dbal6tor35
0s55oba162:672
0:55dba162:604
06558ba168885
0x77616242030
065dtba16711b4
De55dba16200d
0655dh16619th
0655dba161690
06556b1160771
0xff65h2220000
0dffd16243f10
0x7ffc1d241110
0codba162872b
065881159980
Dx5odba1625ofe
0455dba1628/34
06550b1628611
0655eb11629902
0676id1o241778
0cffoid243f10
dofffcdd241fa
1625e
0k5odba161ef37
0x53dba162862
(655601626660
0655eb11629bb8
00ffd10242005
0/771162411288
0r55dba1629798
0555ma15:bedd
0r5tdbal61of34
0s55eba1620644
0:55eb31620dca
06556b11239998
0655dba16158b4
7655ths167907
2555dba169000
Re55dha1616bde
0s55dbw1629948
0655s8s16999110
0655
0csodbs1628113
0653801628/10
De5idbalo2ciee
D655bb1662950
065
065der1612ff
0555da166bbe
0253cha1622897
0655dba16900
065dbr16129
0x7ffc1d241fac
065dbb16250cc
0i530ba1633075
04550ba1620649
0655bb162006
0655dbo16285e1
0665db-167712
0677
077016102411200
865dhh16288611
0555
0s55dha16164448
0s55dba1620128
1655
0555
1.55dba1517711
0/55dba1622da00
dc55dba161be47
0455dba162ccB8
0e55dba1618848
0s55dba1628891
06558b11628500
0655sba1620000
0.55dba1617114
027
155688
1e55dbw162efb6
1s55dhw1623s40
0655sba161666
0655
ooffd1d243fac
0offfe1d24111
0c3dbs1616h32
066880162864
Dx50dba162607
0653eba1620645
0655eb11620ee
0655eba16266e1
0offid102400s
0477016244120
0v71fc1d241115
065dbb26620002
065ddb1526773
0r55dba1618bb4
0:55dba1615a13
1655
0655/b31616621
0677610242010
065d0b1122dd10
0155dba1628acf
0c55dbs161c912
06550b151be52
0v55dbal6188bbb
0+55ebal62:673
0:556b1662:666
0655eba1628:70
0o7755c2220188
0r55dba1617118
0u55diaa150
1529748
1e55dba1622fh?
0s55dba1610083
06558ba1628733
0x776562440000
0x7f
003d6a11228d22
0c3ddb1122752
06298
De53dba102607
D633dh1122836
005569162005
oxffid62241fu
0dffd10243118
0v55dba1617a
065dbb161bb44
01558b1528718
D/55dba16100-4t
04550ba16238b4
0655db116285d
06558b11623700
0c79id1224150
063db31628605
Dx55cba1625acf
065obb1625730
0655bbll5bbel
Dx55dba1616b-4d
D455eba162064e
0155bb16200ee
0k55dbo1629990
0x7Rdd1224198
0/55/bb1667110
0:55dba161711
dc55dha1629478
1655dba162900c
0s55dba162:thb
0s55dha1629748
16550
16557
0u77d162441a0
Dv55cba1678dd2
0e5sdbs1628718
8e55dba1626078
1o55cha1629610
10
06550ba160000
Oxhiidho244fal
0655tb4167711
0155cha16171f
0e55tha161bw40
0555
1655dbw16168b4e
hs55dbw16286d
1155
0655
0633601628664
0658801626/3b
0e5odba101db4f
0653eha162064dd
1653db11620ddd
06536
0655db1617111
0r5dbb:1628617
06558b152:611
0:5tdba162etba
0s55oba1629128
1655001622800
06251
0c76616242018
678
0r55dba167kad
de5ddha161bed)
1s55dba1622079
1s55tha1629611
1605
0655dbba167115
0:55dba1617ifd
de5ddba161le66
0555/
1e$5dba1616650
1655dba1620644
1655a
0655
0.55dba16216644
0c 678
0cSddbs161033
16288b1668711
1e53dba1610851
0653dbw162064t
0offihh22e1eo
065dbo161712
0653dba161716
0c5odb1622o03
06558b1552612
D453dba162erba
0453eba1615af4
06550b1161051c
0658bb1610622
0665dbo1622544
0:55dba1628440
165ddh2661603
0655dba1678bb
1=55dba163a00
1:55dba162::74
165
16558
077651122416888
0655/b1167711
0155dba16171tc
b655db31228400
0v55/ba162:665
0v55sbal616652
0155dba16188b4
1655
16557
0u55dba1615880
0x5idha16700 1
0e5ddha1628721
15986
1e55dha1616655
0s55chw1679654
34
0655s
060db5161715
0offfodd241110
desddbsllbbee
065380016266
De5idba162rfbb
De53eba162853c
05556b11226dd
0655bb1629700
Doffid1624050
063d611628sc
0codbs1129700
De5odha162607b
0250cha1620012
D6530ba1620822
06536a1167990
0655601617114
0d7ffc1d241fe4
065dba1629:50
0c55/ba152c64
0:550ba16188544
Dr55dba1629748
06558b1629710
06558b162871c
0655db1616:500
0077/112241100
0771161d241110
de5ddha6628714
0277516241110
de55dba1616655
ds55oba1629030
1655
1655
0v7rddic24100
0wffidd2211110
0x7ff61d24tfe
besdbsttbbbed 010
besodba102efbe
0s55dba1628051
0055001628065
005091165900
065ddr11ibbee
0229151624199
BesSaba
0s55dhw1600551
0655d8a1600811
0655
x7fdd10241f44
0xfffc1224160
0codba1622616
00ff616242004
04500ba1610556
De53eba1629128
06556b1669044
0655eb1160990
0665db:1616:22
0077111224111
0x7ffcid241fd:
165db216llbef
0x77d11243110
1e5505a1818057
0s55dba1629899
1655
1655
dc5ddbs161bed;
0x7ffd16240088
0v5tdbal62:tbd
0.55eba1620053
01550b1162005
16566
0x773661856888
de55dha1616h54
06771516242188
Re55dba162207d
0s55dba162055)
0655alaa6600000
065Sobba162664
1u55dha15166:00
Figure 28: Part of the logged side channel trace and informative side channel data points marked in red.
31

---

Published as a conference paper at ICLR 2021
Function
Instruction Address
Accessed Memory Address
put_rgh
Ox7f38dab131b8
0x55dba1628acc
put put_rgb
0x7f38dab131f9
0x55dba1628acc
put_
Ox7f38dab131b8
0x55dba1628ac4
put_rgh
0x7f38dab131b8
0x55dba1628ac6
put_rgb
Ox7f38dab13109
0x7ffd1d241f18
put_rgb
Ox7f38dab131fe
0x55dba162cc44
put jur_demapped_et
0x7f38dab13010
Ox7ffd1d241fe4
put put_rgb
Ox7f38dab131e0
0x55dba162cccb
can
Ox7f38daffd6cd
0x55dba161be41
0x7f38daafd6a8
0x55dba1628d5c
0x7f38daafd6a8
0x55dba1628d50
jsimd Saaddd_oooo________
0x7f38daafd6bc
0x55dba1629e80
@ecidd_covvsee
0x7f38daafd6a8
0x55dba1628d50
_aiiidd_oovsamp_____11 float
0x7f38daafd6bb
0x55dba1629e88
float
0x7f38daafd6bb
0x55dba161be62
jsimd jsimd_can___ _acaa_dc_________eee__ee________ is
Ox7f38daafd6cd
0x55dba161be61
put_rgb
0x7f38dab131db
0x55dba16286f5
jsimd can
fdct caa__acn____aa_
islow
Ox7f38daffd6cd
0x55dba161ee25
@eiidd_can__d fdctt islow
0x7f38daffd6cd
0x55dba161ef2d
jsimd @simd_can___ cac_ca__ffo_ Sacn___ct_____ islov
0x7f38daafd6cd
0x55dba161ef35
output_ppa
0x7f38dab10293
0x55dba162cfb7
output_ppp
0x7f38dab10299
0x55dba162cfb9
@sind_convsamp float
Ox7f38dab10260
0x55dba1616b50
float
0x7f38daafd6b5
0x55dba1620be2
isimd fdet islow
Ox7f38daafd6cd
0x55dba1620c39
0x7f38daafd6a8
0x55dba1628d34
jsimd_ @siid_convsampp_leg float
Ox7f38daafd6bc
0x55dba1629f30
jsimd can fdet islow
0x7f38daafd6cd
0x55dba1620de1
[eiind_convsagg float
float
0x7138daafd6b5
0x55dba1620de6
jsimd can_ fon_fdct_isloo
0x7f38daafd6cd
0x55dba1620efd
Ox7f38dab03f1d
0x55dba1616c14
Figure 29: Isolated functions in 1ib ipeg that contribute to the reconstruction of user private im- ages.
32


Reviews:
[{"review_id": "gFY5H6_Pls", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Objective of the paper:  \nThe objective of the paper is to show that the side channel information (based on timing, etc.) for image analysis software can be used to reconstruct reasonably accurate to highly accurate approximations of user images.  While this was previously known, in this paper, the authors use machine learning techniques to more efficiently \"learn\" how to use this side-channel information, developing generative models that allow image reconstruction.  \n\nStrong Points:\n1)  The use of machine learning techniques as an \"attack methodology\" for making use of side channel information seems to be a potentially powerful methodology.\n2)  The results shown here seem quite strong.  \n3)  The application (image analysis) is well chosen;  the side channels under consideration seems quite plausible.  \n4)  The code is made available.  \n\nWeak Points:\n1)   There are obvious counters to this approach;  once one knows that this type of software is being targeted, one can introduce various randomization (for memory accesses, timing, etc.) to lessen the power of side channel attacks.  That is, this may only be a reasonable attack because nobody has to this point considered it important to avoid side channel information being released by image analysis software.\n2)   It's a little unclear (to this reviewer) how much \"information\" the learning algorithm is seeded with.  That is, for these image sets, how similar/distinct are the underlying images?  Your examples seem to capture a lot of the original image, but it's not clear to me how to interpret what the algorithm is starting with.  (Essentially, would these results scale to much larger sets of images?  How much does working with the limited set of celebrity faces already help yield the strong results?  This is tackled somewhat in Appendix F, so the authors do seem aware of this point.)\n3)  It seems like you're considering all the different side channels separately?  Is there any way to combine the multiple side channels (into a mega-side-channel?) and do better?  (I apologize if I'm missing something -- I'd expect to see a result set with each side information, as you've given, but then results with all combined.)\n4)  This reviewer does not have good insight into why this type of side information would be useful in determining how to reconstruct images in the manner done here.  Some description or intuition as to how accessed cache lines could intuitively be used to determine an approximation to the original picture would be welcome.  (Understood you have limited space.)  \n5)   The paper does seem like a first step and there remains significant room for improvement (improved side channels, improved scalability, other applications).  \n\nOverall Rating:  \nDespite the (relatively minor) weak points, the paper seems to be a strong proof of concept that machine learning can greatly increase the risk of side channel attacks, because they can essentially automate the task of determining how to use the side channel information to obtain the desired result.  This seems to be an important message for the community, in particular as I expect this framework is not specific to image analysis software, but can be applied more generally.  \n\nQuestions for Authors:\nI would very much like to see some sort of experiment where noise is injected in the side channel information.  One can view this as a setting where one obtains limited side channel information for some reason, or as a setting where the program is modified in some way to lessen the amount/value of side information.   \n\nSimilarly, as discussed above, is there some intuition as to why the side information yields such reasonable results?  (Again, and how this relates to the limited data sets?)  And whether or not you could combine multiple types of side information naturally using this approach?\n\nOther Feedback:  \nOverall thee paper seems well written, and you do provide a great deal of data.  ", "extra_scores": {}, "extra_text": {}}, {"review_id": "_xlddZXghlz", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper describes a  new side channel analysis attack framework to reconstruct images from system side channel. This new framework consists of two submodels: the VAE-LP model generates a coarse reconstruction image and the GAN model refines this reconstruction. The experiments show good results in reconstruction.\n\nThe idea of reconstructing images from system side channels is interesting and the proposed approach shows good performance. I have the following concerns about the publication of this paper:\n\n1. The novelty is limited from the perspective of the method. The VAE with learned prior is new but little discussion is provided that why a learned prior is necessary. Can we just use a simple supervised end-to-end autoencoder to learn the mapping from side channel signals to the image? Can we use a standard VAE without the Image Encoder branch? Btw, these two are the important baseline for comparison. The authors need to explain why such a design is necessary, or better. Is this a framework specifically designed for SCA, or is it suitable for any task of image reconstruction from any signals?\n\n2. The motivation of the GAN submodel is confusing. As shown in Figure.3, the results of VAE-LP are blurry and are difficult to identify the details, which is reasonable as the input may include very limited information. The GAN submodel adds a lot of details to these images. Some of them are far from the reference images, which is also reasonable as GAN tends to add **texture-like noises**. The refinement or the rich structures and textures are made-up by GAN for good **visual effects**. This additional information is not guaranteed to be correct. Usually, GAN is applied where we require good visual effects. Do we need this in SCA, even more important than the accuracy of the results?\n\n3. The GAN submodel also faces the challenge of **model collapse**, which refers to the problem that the network tends to generate limited patterns to fool the discriminator. As the training dataset is highly structured and the patterns are relatively simple. Is the proposed method capable of working in the scenario of reconstructing natural images? Is the risk of model collapse a concern for the SCA task? This discussion deserves to be included in the paper.\n\n4. Another question that may attract the attention of the machine learning field is, can we know what information in the side channel is used for image reconstruction? In the paper, the side channel information is folded into 2D matrix because of the spatial dependency. Can we visualize this dependency? Is it robust to noise or shifting?\n\n--- Updates ---\n\nAfter reading the authors' response, I decide to raise my rating to \"accept\".\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Vv2x7zT8CB", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors proposed a representation and generation model to reconstruct the image signals from system side channel signals. The task itself is interesting and novel, demonstrating the first efforts and impressive performance on recovering noisy side channel signals. The work will potentially inspire more attempts to conduct attack and security research on computer internal signals. \n\nPros:\n\n+ A clear problem statement and description of the proposed method, and very impressive performance in both visualization and quantitative metrics.\n+ A good analysis of the generalization ability of the model.\n\nCons:\n- The method itself makes sense to me, but not novel. The authors propose to share latent codes between two modalities and train them jointly. And then use GAN to further refine the image quality for visualization. The overall pipeline is very common in image restoration and generation area. \n- The 'learn the prior' method may fail to learn a good latent space without another prior distribution regularization. It's better to compare it with first training the VAE with image encoders using fixed gaussian prior, and then train the latent space jointly while fixing the image encoder. I believe the latent space will be better regularized and smooth. However, the tasks do not require you to generate image from the latent space, the advantages may not be revealed from the reconstruction results.\n- The performance of the model trained on mini-ImageNet is poor, even though the authors state that it still reconstructs the basic structures. However, it also reminds us that the good reconstruction results are highly related to the small image training database domain. It may be harder for larger databases or other unseen data. \n- Will the organization of the matrix influence the results? What if there are random zero paddings or other errors in the log?\n- It's better to analyze and think about the possible solution to avoid these kinds of attacks. Otherwise, purely talking about the attacks strategies may raise some ethics concerns of the paper.\n\nIn summary, the task sounds interesting to me, but the overall method is not novel at all. The authors utilized a totally old technical model and training method in a brand new topic by formatting the input modalities. The results seem impressive in smaller dataset, but it still not convincing enough when testing the model on cross-domain data. Besides, there are possible ethics issues if no discussion related to avoiding the attacks is covered. \n\nUpdate:\nThanks for all the authors' feedback and the make-up experiments as well as revisions. I appreciate the authors' efforts on solving the privacy problem and new input forms, but I really think the paper made no contributions to 'learning representation'.  The VAE and GAN-based model is not problem-specific or novelly designed for the task. Instead, it is a general framework for any reconstruction problems. I do believe this will make a strong submission to other conferences related to information. In this case, I will not change my rating.", "extra_scores": {}, "extra_text": {}}, {"review_id": "8BBaGFkHmtU", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n#### Summary:\nThe authors present a framework that uses a combination of   VAE and GAN to recover private user images using Side channel analysis of memory access . A VAE-LP model first reconstructs a coarse image from side channel information which is reshaped and processed using a convolutional network. The output of the VAE-LP model is refined using a GAN to add fine details. Compelling results are demonstrated for recovery of private information and state of art metrics are reported. \n\n#### Strengths:\n1. The paper is well written with great attention to detail. All design decisions are well motivated and have been adequately explained.\n2. State of the art qualitative results are demonstrated for recovery of images using SCA .\n3. The paper is an extremely interesting application of the VAE-GAN framework. \n4. The proposed solution offers a way to recover private information without requiring specialized knowledge of the system architecture, which is a great advantage as it is potentially generalizable to a wide variety of architectures.\n5. To the best of the reviewer's knowledge, the treatment of related work is adequate and the positioning of the current work in the context of prior art is made very clear.\n6. Analysis is performed on a wide variety of datasets indicating that the solution is not over-engineered to a particular library or architecture. \n7. The appendix presents a lot of interesting experiments and analysis. Particularly, classification performance and recovery quality on images trained using a more generic rather than domain specific dataset. \n\n#### Weaknesses:\n1. Although the authors mention that direct comparisons are hard to provide to previous methods due to lack of public implementation. Few qualitative examples of recovered images from prior art (relegated to the appendix) would help motivate the need for the proposed approach better.\n2. It is not clear why the L1 loss is necessary for the second stage in section 3.3 . Particularly, as opposed to a more well studied framework like Pix2pix[1], which imposes a L1 loss at the output with a ground truth image. Because at training time we know the GT image that we are recovering. An ablation study demonstrating the quality without this L1 loss would be insightful.\n3. Reporting image level metrics is instructive in understanding how far the reconstructed image is from the ground truth image. Particularly, SSIM and LPIPS[2] score of generated image w.r.t ground truth image. \n4. Although the formulation is interesting. The novelty is somewhat limited as it is straightforward application of the VAE-GAN framework to SCA data.\n\n#### Questions:\n1. The exact nature of the trace encoding is unclear. Are memory locations fed directly into the network? If so how are the normalized, because assuming 16 bit addresses, the range of values occupied \n2. The authors mention that libraries like libjpeg generates 700k side channel \"signals\"  in Section 1. What is the nature of these signals? Particularly are the single bits of information/ text/ memory locations? \n3. Although beyond the scope of the presented work, this framework allows for access of private information. Such methods raise obvious ethical concerns on part of services which offer ML solution. A small statement about how this form of attack can be protected against would be instructive in strengthening the narrative. \n4. The motivation to use a CNN to process trace matrix is unclear. Particularly, are assumptions of equivariance made on the access patterns?  Apart from the architectural convenience of weight sharing is there a stronger motivation to assume a convolutional structure on the input? An MLP(or 1*1 conv) with pooling run on the same $$N \\times N \\times K$$ encoded trace matrix would provided some insights regarding the need for a CNN to process the trace information. \n5. It is not clear what the generated *Dec* refers to in Sec. 3.2.\n\n#### Minor:\nsubsumed -> assumed ? in section 3.2 \n\n#### Justification of rating:\nAlthough the authors present an interesting application of the VAE-GAN framework for recovery of private information from SCA, the formulation of the problem is remarkably similar to conditional-VAE. It would be interesting to further analyze the learnt representation from the VAE-LP to see if certain \"code-blocks\" or functions can be isolated. Having said that, the paper is very well written and thorough in its analysis. The reviewer also believes that a systems/privacy audience would be able to appreciate the contributions of this work better.\n\n#### Updates\nIn light of the responses provided by the authors, and the thorough nature of the additional experiments I am inclined to increase the ratings of this paper. The followup responses answer most concerns raised by reviewers and provide supporting evidence. However, the reviewer maintains that this work could potentially be of more interest to a privacy audience who may better be able to appreciate the use case considered in the presented work.\n\n[1] Isola, Phillip, et al. \"Image-to-image translation with conditional adversarial networks.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.\n[2] Zhang, Richard, et al. \"The unreasonable effectiveness of deep features as a perceptual metric.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper shows that it is possible to reconstruct private images from CPU cache line and OS page table accesses side channels, using a generative model on top of side channel traces. The reviewers agree that the problem is interesting and the experimental evaluation makes a convincing case that such an attack is possible. The author rebuttal was useful in clarifying some aspects of the paper, and the discussion on possible mitigation strategies is a nice addition to the paper.
Decision: Accept (Poster)