Title: Penguin: Parallel-Packed Homomorphic Encryption for Fast Graph Convolutional Network Inference
Venue: NeurIPS 2023
Accepted: yes
Keywords: Cryptographic inference

Graph Convolutional Network

Parallel Packing
Abstract: The marriage of Graph Convolutional Network (GCN) and Homomorphic Encryption (HE) enables the inference of graph data on the cloud with significantly enhanced client data privacy. However, the tremendous computation and memory overhead associated with HE operations challenges the practicality of HE-based GCN inference. GCN inference involves a sequence of expensive matrix-matrix multiplications, and we observe that directly applying the state-of-the-art HE-based secure matrix-matrix multiplication solutions to accelerate HE-GCN inference is far less efficient as it does not exploit the unique aggregation mechanism of two-dimension graph node-features in GCN layer computation. 
As a result, in this paper, we propose a novel HE-based ciphertext packing technique, i.e., Penguin, that can take advantage of the unique computation pattern during the HE-GCN inference to significantly reduce the computation and memory overhead associated with HE operations.
Specifically, Penguin employs (i) an effective two-dimension parallel packing technique for feature ciphertext with optimal graph node partitioning and graph feature interleaving, and (ii) an interleaved assembly technique that can effectively make use of the blank slots to merge ciphertexts after feature reduction and significantly reduce the costly rotation operation.
We provide theoretical analysis and experimental validation to demonstrate the speedup achieved by Penguin in accelerating GCN inference using popular GCN models and datasets. Our results show that Penguin can achieve up to $\sim10\times$ speedup and around $\sim79$% reduction in computational memory overhead, significantly outperforming state-of-the-art solutions. To the best of our knowledge, this is the first work that can ensure the protection of both graph structure and features when accelerating HE-GCN inference on encrypted data. Our code is publicly available at https://github.com/ranran0523/Penguin.

Penguin : Parallel-Packed Homomorphic Encryption for Fast Graph Convolutional Network Inference

Abstract

The marriage of Graph Convolutional Network (GCN) and Homomorphic Encryption (HE) enables the inference of graph data on the cloud with significantly enhanced client data privacy. However, the tremendous computation and memory overhead associated with HE operations challenges the practicality of HE-based GCN inference. GCN inference involves a sequence of expensive matrix-matrix multiplications, and we observe that directly applying the state-of-the-art HE-based secure matrix-matrix multiplication solutions to accelerate HE-GCN inference is far less efficient as it does not exploit the unique aggregation mechanism of two-dimension graph node-features in GCN layer computation. As a result, in this paper, we propose a novel HE-based ciphertext packing technique, i.e., Penguin , that can take advantage of the unique computation pattern during the HE-GCN inference to significantly reduce the computation and memory overhead associated with HE operations. Specifically, Penguin employs ( i ) an effective two-dimension parallel packing technique for feature ciphertext with optimal graph node partitioning and graph feature interleaving, and ( ii ) an interleaved assembly technique that can effectively make use of blank slots to merge ciphertexts after feature reduction and thus significantly reduce costly rotation operations. We perform detailed theoretical analysis to support our arguments. In the meantime, our experimental results also show that Penguin can achieve up to ∼ 10 × speedup and around ∼ 79% reduction in computational memory overhead, significantly outperforming state-of-the-art solutions. To the best of our knowledge, this is the first work that can ensure the protection of both graph structure and features when accelerating HE-GCN inference on encrypted data. Our code is publicly available at https://github.com/ranran0523/Penguin .

1 Introduction

Graph Convolution Neural Networks (GCNs) have recently demonstrated phenomenal performance for many privacy-sensitive applications such as social networks [34], cross-domain recommendation systems [35], and personal healthcare [18]. A popular solution for clients seeking to leverage these advanced GCN models is to utilize cloud-based inference services. However, clients often hesitate to share their graph data with the public cloud due to concerns about sensitive information, such as graph structure and node features that reveal personal social relationships and medical records. To address this privacy concern, one viable approach is to adopt the Homomorphic Encryption (HE) scheme [3, 6, 7]. By performing the entire inference computation on the cloud using encrypted data,

Tao Liu

Wujie Wen

Latency

(s)

Latency(ms)

Rotation

PMult

Add

6.2

0.297

0.318

6.44

CMult

Rotation+CMult

PMult+Add

Non-Linear

(a)

(b)

Linear

96.70%

99.98%

Linear

Latency

Total

Latency

(c)

Latency

Memory

Feature-wise

Relative Ratio

Node-Feature-joint

1.0

0.76

0.21

0.35

0.09

the privacy of client data is significantly enhanced. This enables privacy-preserving GCN inferences while ensuring that sensitive information remains confidential.

While the idea of embedding HE into GCN inference on graph data seems appealing, it faces several significant challenges: Firstly, similar to HE-based CNN inference on non-graph data (such as for using convolutional neural networks (CNN) [10, 4, 8, 23, 2, 19, 26, 17, 14]), the enhanced privacy would come at the cost of the tremendously escalated computational overhead associated with HE operations (e.g., ciphertext (ct) rotations/multiplications, additions), which could be orders of magnitude higher than the counterparts in the non-encrypted computation [10, 29, 16]. Secondly, existing solutions focusing on alleviating computation overhead of HE-based CNN inference may not be applicable or optimal to GCNs due to computing pattern differences between the CNN and GCN [21]. For example, a GCN layer's computation is dominated by the special consecutive matrix multiplications ( A · X · W ) for 2-dimensional feature-node aggregation-feature aggregation via multiplying a high dimensional feature matrix X with weight matrix W , followed by graph node aggregation with A , while a CNN layer's computation is bottlenecked by multi-channel 2D convolutions. Thirdly, simply treating the above critical matrix operations in HE-based GCN inference as a traditional encrypted matrix-matrix multiplication (MM) problem for speedup is sub-optimal because: 1) state-of-the-art (SOTA) HE-based MM acceleration often requires the matrix to satisfy some special properties, e.g. square matrix with size 64 × 64 [15], while GCN matrices like feature matrix X are typically irregular depending on applications (i.e. 2708 × 1433 in Cora dataset [32]); 2) SOTA solutions focus on a one-time MM without considering the consecutive MMs incurred by the two-dimensional feature-node aggregation, as well as the further processing of MMs' result in the next GCN layer. This leads to inefficient ciphertext space utilization and unnecessary HE operations, which further translates into prolonged HE-GCN inference, as we shall show in Sec. 4.2.

To better understand the computation cost of HE operations that dominate the HE-GCN inference latency, we profile the latency of different HE operations using one GCN layer with 32 hidden units and the Cora dataset with 2708 graph nodes and 1433 (32) input (output) features per node. All HE operations are defined in Sec. 2. For generality, we assume both feature matrix and adjacency matrix are encrypted , which is a typical case in inductive learning (e.g. dynamic graph structure in link prediction) [22]. Without loss of generality, the same indexed features from different nodes are packed as a ciphertext (feature-wise packing) and the encrypted matrices are diagonal-encoded for MMs (detailed settings in Sec. 4.1). As Figure 1 (a) shows, first, the latency of ciphertext rotation and ciphertext multiplication (CMult) can be much higher than other operations like plaintext (pt) multiplication (PMult) or Addition, e.g. > 20 × Rotation v.s. PMult. Furthermore, about > 99% latency comes from the linear operations (mainly HE rotation and CMult due to the consecutive MMs), instead of the nonlinear operations (ReLU replaced by a square function) due to feature reduction in GCN (from 1433 input features to 32 output features). Meanwhile, for linear latency, Rotation and CMult dominate the latency (e.g. > 96% of total) as the size of the adjacency matrix could be quite large (Cora: 2708 × 2708) in the GCN problem. Last, we profile the latencies of different ciphertext packing formats under the same evaluation setup as (b) in Figure 1 (c). From the profiling result in Figure 1 (c), either the node-wise packing format (e.g. 1 ciphertext contains one node's 1433 features) or the feature-wise packing format (e.g. 1 ciphertext contains the same indexed features from 2707 nodes) could not effectively perform the HE-GCN inference. With node-feature-joint

packing format (e.g. 1 ciphertext packs 32 features and 128 nodes) by our proposed Two-Dimension Parallel-Packing (see Sec. 3.2), the ciphertext size is fully exploited, and the total HE operation count reaches a minimum, leading to significantly reduced latency and memory cost. These results indicate that the key to accelerating the HE-based GCN inference is to significantly reduce the rotation and CMult operations with a GCN-dedicated ciphertext packing format.

To this end, we propose Penguin , a novel HE ciphertext packing framework dedicated to accelerating GCN inference with the consideration of encrypting both graph structure and features simultaneously (both adjacency matrix A and input feature matrix X ). The driving vision of Penguin is: feature ciphertext packing ( X ) for efficient HE-based GCN inference needs to be designed in a manner that is aware of the unique GCN computation-both the left-side graph node aggregation AX and right-side feature aggregation ( XW ), instead of optimization in one direction (either AX or XW ). In this way, the whole ciphertext space can be efficiently utilized with minimized slot waste, enabling the significant reduction of ciphertext number (memory overhead) as well as the expensive HE rotation and CMult operations under the single instruction multiple data (SIMD) architecture. Our major contributions are three-fold: 1) We propose an efficient two-dimension parallel packing technique for ciphertext via optimal graph node partition and feature interleaving. By performing the feature-level aggregation first and formulating the HE computation overhead as a constrained optimization problem, we analytically obtain the best feature-node partition that can maximize the usage of ciphertext space and minimize the costly HE operations. Experimental results are well consistent with theoretical analysis. 2) We propose an interleaved assembling (IA) technique to efficiently merge ciphertexts with blank slots incurred by feature dimension reduction in the feature aggregation stage. This extra-level optimization further significantly reduces the number of ciphertexts and associated HE operations. 3) We comprehensively evaluate our proposed Penguin for CKKS-based GCN inference using Cora-based graph node classification, Citeseer-based link prediction, and Pubmed-based link prediction. Results show that our method achieves by up to about 10 × inference speedup and 79% memory overhead reduction, significantly outperforming the state-of-the-art solutions. To the best of our knowledge, this is the first work focusing on accelerating the HE-based private graph convolutional neural network inference on encrypted graph data, of which both the sensitive graph features and graph structure are protected.

2 Preliminary

CKKS Homomorphic Encryption Scheme. Homomorphic Encryption (HE) allows computations on encrypted data. HE has different categories according to the different computation types they support. The Leveled HE (LHE) schemes support a limited number of additions or multiplications while Fully HE (FHE) allows an arbitrary number of computations using a bootstrapping procedure that can effectively refresh the ciphertext and obtain a new ciphertext that encrypts the same value but has lower noise [9]. In this work, we focus on reducing the number of bottlenecked operations in CKKS-one of the promising LHEs, without considering the costly bootstrapping.

CKKS [6] is an LHE scheme and its security is based on the hardness of ring learning with errors (RLWE) problem. CKKS allows arithmetic operations on encrypted data over fixed-point numbers with predefined precision, which makes it an ideal candidate for performing machine learning tasks where most of the computations are approximate. The supported homomorphic operations include ciphertext addition Add ∼ ( ct 1 + ct 2 ) , ciphertext multiplication CMult ∼ ( ct 1 × ct 2 ) , plaintext multiplication PMult ∼ ( ct × pt ) , ciphertext Rotation ∼ ρ ( ct, k ) . The rotation is to apply Galois automorphisms of the cyclotomic extension to the plaintext polynomials in encrypted form resulting in a cyclic shift of the slot vector. Among these four operations, Rotation and CMult are substantially slower ( ∼ 20 × slower) than ciphertext-plaintext addition and multiplication as shown in our runtime performance of CKKS in Figure 1 due to the expensive key-switching operation [23].

Graph Convolution Neural Network. To extract the hidden graph features H , the 2-dimensional feature-node aggregation of a typical GCN layer can be often abstracted as [21]:

Where X ∈ R N × F is the input feature matrix. W j ∈ R F × F ′ represents weight parameters to transform the input features from an input dimension F to an output dimension F ′ ( feature level aggregation). ˜ D j . ˜ A j is the adjacency matrix with self-loop. The XW term is implemented by

1

1

1

0

0

0

0

0

0

0

0

1

0

0

0

0

Feature-Aggregation

Node-Aggregation

a fully-connected (FC) layer (node level aggregation) and then multiplied with the normalized adjacency matrix ˜ D j -1 2 ˜ A j ˜ D j -1 2 . Finally, a non-linear activation function σ (e.g. ReLU) is applied to get one GCN layer's output feature matrix H . Throughout this work, we refer A as the normalized adjacency matrix since normalization could be absorbed in a pre-processing step.

Threat Model. We adopt a threat model setting consistent with prior works [10, 15, 4, 8, 19, 23, 28]. A client uploads private and sensitive data to the cloud for obtaining the online machine learning model prediction results. The cloud server is semi-honest (e.g. honest but curious). To ensure data privacy, the client encrypts their own data by HE and decrypts this inference result by their private key. In this work, we focus on encrypting both graph node features X and the normalized adjacency matrix A . The clients run the decoder of GAE [22] at their end because this step does not involve trained model parameters on the cloud server.

3 Method

Overview. The GCN inference A · X · W can be separated into the two-dimension ( A · X on the nodes and X · W on the features) aggregation on feature matrix X . When we perform HE matrix multiplication on the encrypted feature matrix (ciphertexts), it is inevitable that we need to perform HE rotation on the same ciphertext. Unfortunately, the rotation operation not only incurs high latency but also generates a huge number of ciphertext copies that consume a large amount of memory space. In this section, we propose a holistic solution set to systematically address these issues.

In order to effectively reduce the number of ciphertexts involved in HE computation, our design is built upon the feature-wise packing since multiplying W often leads to a lower feature dimension. However, for non-densely packed ciphertexts, feature-wise packing is further subject to the data alignment issue, resulting in extra rotations. To overcome this challenge, we propose the twodimension parallel-packing. In addition, considering that the layer-wise feature number reduction would result in many wasted slots, we further propose the interleaved assembling to efficiently merge such ciphertexts.

3.1 Motivation of Feature-Oriented Ciphertext Packing

The major inference computation in GCN can be illustrated as A · X · W , where A ∈ R N × N is the normalized adjacency matrix used for node-wise aggregation, X ∈ R N × F is the input feature matrix, and W ∈ R F × F ′ is the weight matrix used for feature-wise aggregation. Apparently, we can choose A · X or X · W as the first step, which will not change the final product. However, considering that matrix X is encrypted as ciphertexts, the order of computation will affect the efficiency since the ciphertexts with fewer dimensions will reduce the required HE operations and copies of the ciphertexts. For example, if F ′ < F , we first perform X · W to produce an intermediate product with fewer dimensions, i.e., R N × F ′ . This will reduce the computational overhead and latency in the next step A · X . On the contrary, if F < F ′ , we first perform A · X .

We explore two ciphertext packing design options that could lead to minimized computational overhead of a single-dimension aggregation (either graph node or feature). One is the feature-wise packing, where one ciphertext only packs one feature data from different nodes. The number of ciphertexts is proportional to feature number. The other is the node-wise packing, where the number of ciphertexts is equal to the number of graph nodes. However, in this case, the number of graph

0

1

0

1

0

1

0

1

1

0

1

0

1

0

1

0

(a)

Interleaved Assembling

(b)

1

0

1

0

1

0

1

0

0

1

0

1

0

1

0

1

nodes (or ciphertexts) does not change during inference, this inevitably results in too many wasted empty slots in the ciphertexts and thus would yield more HE rotations. As the example in Figure 2 shows, we assume the ciphertext packing size is 8, adjacency matrix A ∈ R 4 × 4 , weight matrix W ∈ R 4 × 1 , and feature matrix X ∈ R 4 × 4 . The 4 feature-wise packing ciphertexts can be reduced to 1 ciphertext after feature aggregation, which only needs 4 ciphertext-multiplication (CMult) in the next adjacency matrix multiplication. If using the node-wise packing, we still need 16 CMult in the next step. Therefore, we choose feature-wise packing in our design.

3.2 Two-Dimension Parallel-Packing

Optimization Problem Formulation Following the definition in the previous section, we assume that the ciphertext has M available slots for packing data and consider the following general case: M > N and M > F . As shown in Figure 2, after the feature-wise packing, the same features of different data from feature matrix X are encoded into the same ciphertext X = [ ct 1 . . . ct F ] and then multiplied with weights w ij to get matrix X ′ = XW = [ ct ′ 1 , . . . , ct ′ F ′ ] in the same feature-wise packing format (see Eq.( 2)).

For node-wise aggregation, we need to perform diagonal-encoded matrix multiplication [12] on ciphertexts F ′ individually. However, due to M > N , we need to generate the corresponding ciphertext copies with each data aligned by rotating each ciphertext twice and scaling with the mask vector then summing (see Eq.( 3)).

After that, we multiply them with the corresponding diagonal-encoded vector d i of adjacency matrix A and sum up them to get the node-wise aggregation result of A · X ′ (see Eq.( 4)).

In this process, we can find that data alignment issue for non-densely packed ciphertexts leads to extra rotations. We propose the two-dimension parallel-packing to solve it. Our idea is to leverage the matrix partition to fully pack data in all slots and amortize the HE computation cost.

Figure 3(a) shows our basic idea. We partition n (a power two number) graph nodes into a small block to fully utilize the size of the ciphertext and encode the feature matrix X . In each ciphertext, we actually pack n nodes corresponding to f different features together in an interleaved way. For feature-wise aggregation, we adopt the baby-step algorithm [19, 15] to get the different output features with good alignment. As shown in Figure 3 (a), the number of rotations 2( f -1) used here for each ciphertext depends on the number of different features f = M/n (we assume the output feature F ′ ≥ f ). Then, we continue to rotate each ciphertext for n -1 times and perform the diagonal encoded matrix multiplication [12] for A · X . The total complexity of rotation is:

Since M = n · f , the total complexity is further equal to

Where n = 2 f , the total complexity of rotation reaches a minimum. Hence minimizing the number of rotations can be modeled as an optimization problem:

Supporting Large Graph Eq. 8 assumes that the number of graph nodes should be smaller than that of ciphertext slots ( M > N ), however, for scaled graph networks, it is possible that M ≤ N . For example, the PubMed [32] contains 19717 nodes, which is far more than the 4096 ciphertext slots. To address this, our method can be scalable to such cases by splitting a large graph into several sub-graphs. Assuming we use feature-wise encoding and it requires 5 cts (each with 4096 slots) to pack 1 feature. Accordingly, each feature will have 4 cts with fully packed 4096 nodes and 1 ct with partially packed 3333 nodes. To solve the problem under the constraintM ≤ N , we split N as:

where x = N mod M , R = N % M . Eq. 9 leads to one R × F sub-block matrix and x of M × F subblock matrices. For the R × F matrix, we refer to Eq. 7 to optimize the Rotation as N = R < M . For other M × F sub-block matrices with N = M , we change the assumption from N < M to N = M . Then, again with our proposed Two-Dimension Parallel-Packed ct, the total complexity of rotations becomes:

The corner case f = 1 is different from that in the discussion of optimization problem formulation. Because ct here is fully packed for n = M and does not have the data alignment issue. For n = M,F = 1 , the total complexity of Rotation becomes:

Except for this corner case n = M,f = 1 , the total complexity reaches a minimum when n = 2 f . We compare the previously proved minimum with the corner case n = M,f = 1 here, and get the difference of rotations complexity as follows:

In general, since M is set as ≥ 2 11 to guarantee security level [17, 28], the above inequality 12 always holds. Thus, when considering a M × F matrix, the proposed Two-Dimension Parallel-Packing can still reach the minimum at n = 2 f .

3.3 Interleaved Assembling

In GCN inference, the reduction in feature size may result in wasted slots in two-dimension parallelpacking. As the example shown in Figure 3(b), we optimally encode 32 different features into one ciphertext at the beginning. After the feature extraction layer with 16 hidden units, the previous dense encoded ciphertext will have half of the slots turn to blank. These blank slots in ciphertext bring higher memory overhead, especially given that the adjacency matrix A is also encrypted, resulting in more CMult operations thus computational overhead. We propose interleaved assembling to solve this issue. Figure 3(b) shows our idea. We rotate the ct 2 that contains node 5-8's features by 1 slot and then add it with ct 1 that contains node 1-4's features. After that, we have a new ciphertext ct ′ that contains 8 nodes with 1 feature. Meanwhile, we multiply the two mask vectors with the ciphertexts for sub-square matrix A 11 (for node 1-4) and A 22 (for node 5-8) and get an interleaved assembled ciphertext contains matrix A 11 and A 22 . Then, by rotating ciphertext ct ′ 3 times and performing element-wise multiplication with new diagonal-encoded ciphertexts of the matrix A 11 and A 22 , we could get the results-ciphertext ct ′ 1 of node 1-4 with the matrix A 11 and node 5-8 with the matrix A 22 simultaneously. In this way, the complexity of HE operations including both rotation and ciphertext multiplication can be reduced by half. After that, we repeat the steps to perform multiplication on ct ′ and ciphertext that contains A 21 and A 12 to get the ct ′ 2 . Based on the formula ct ′′ = ct ′ 1 + ρ ( ct ′ 2 , 1) , we get the final result ciphertext ct ′′ that multiples matrix ( A 11 , A 12 , A 21 , A 22 ) . By leveraging such an interleaved assembling, we could achieve f ′ f times reduction of the total computational complexity, where f ′ is the number of features on the current ciphertext, and f is the number of features on the ciphertext before feature reduction.

4 Evaluation

4.1 Experiment Setup

Datasets. We adopt the Cora [32], Citeseer [11] and Pubmed [32] scientific publication datasets for graph learning. The Cora, Citeseer, and Pubmed contain 2708, 3327, and 19717 publication nodes divided into 7, 6, and 3 classes respectively. And each node consists of 1433, 3703, and 500 unique word features, respectively. To test the link prediction task [22], 90% of edges are removed and all node features are retained on all datasets.

Models. We train 3 Graph Auto-Encoder (GAE) models with 2 hidden layers and 2 activation layers on 3 different datasets, i.e., Cora, Citeseer, and Pubmed. The three models follow the same GAE architecture in [22], and are implemented using the DGL library [33]. Table 1 lists the model architecture and pertinent encryption parameters for encrypting both adjacency matrix A and feature matrix X . We use x 2 as the non-linear function [10] to replace the ReLU activation and apply the ADAMoptimizer to train the model for 200 epochs using a learning rate of 0.01. The accuracy of each model (AUC in Link Prediction) is maintained at the original level.

Encryption parameters. For all tasks, we apply a scaling factor ∆ = 2 30 to ensure the accuracy of the encrypted inference using CKKS. Each rescale consumes 30 bits of ciphertext modulus Q , and there are 6 times rescale and corresponding 6 levels across the whole network. Thus, we set Q = 218 , and the polynomial degree N = 2 13 to guarantee a 128-bit security level. Additionally, the scale factor of mask plaintext used in comparison with E2DM [15] & uSCORE [13] is set to 2 15 .

Baseline designs. To better evaluate the proposed approach, we develop several baselines, including:

Penguin-family. We implement several Penguin baselines by applying only our proposed twodimension parallel-packing technique (see Sec. 3.2). We set up different pairs of features and nodes when optimizing the packing format. Table 2 lists the numbers of features/nodes selected. Here Penguin ( f, n ) denotes that f features and n nodes are used in the corresponding baseline design. Note that, the baseline designs with f = 1 or n = 1 are the extreme cases when only the feature-wise or node-wise packing method is used.

Penguin+IA. We develop two Penguin+IA baselines by further applying the proposed Interleaved Assembling (IA) technique (see Sec. 3.3) to the Penguin-family.

We also implement a set of state-of-the-art secure matrix matrix multiplication solutions, which include Gazelle [17], HElayers [1], E2DM [15] and uSCORE [13]. To ensure a fair comparison, we do not apply the activation pruning technique for multiplicative level reduction across all implementations [28], as this technique is orthogonal to our proposed solutions.

Measurements. We use inference latency as our main performance metric, which is averaged over 20 simulations. Besides, we record the Homomorphic Operation Count (HOC), including the number of rotations ( Rotation ), the number of ciphertext multiplications ( CMult ), etc. We also calibrate the numbers of ciphertexts and memory usage. A lower number of these metrics indicates better performance.

Environment. We conduct all experiments on a machine equipped with Threadripper 3975WX CPU using the single thread setting to test the inference latency and train these GAE models with 2 Nvidia 3090 GPUs. We use Microsoft SEAL version 3.7.2 [31] to implement the RNS-variant of CKKS [5] scheme.

4.2 Evaluation Results

4.2.1 Two-Dimension Parallel-Packing

Table 2 presents our evaluation results of the proposed two-dimension parallel-packing and interleaved assembling approach. We find that the packing format Penguin ( f, n = 1) performs the worst on the three datasets due to having the largest number of HOCs and no slot packing optimization. This results in significant latency and memory overhead. In particular, since PubMed contains more encrypted features (number of cts), the same design performs worse on PubMed than on the other two datasets. We use this Penguin ( f, n = 1) as the baseline to compare the speed of other approaches.

Our results clearly show that our proposed two-dimension parallel-packing method can significantly reduce the HOCs (especially the number of rotations) and the number of ciphertexts. For example, on the Cora dataset, our Penguin (16 , 256) , Penguin (32 , 128) , and Penguin (64 , 64) designs can reduce the number of rotations from 1048K to 9.7K, 8.3K, and 13.5K, respectively, thus reducing memory usage by ∼ 79% , ∼ 76% , and ∼ 47% , and reaching ∼ 10 . 35 × , ∼ 8 . 06 × , and ∼ 4 . 25 × speed up, respectively.

In particular, the results we observed are well consistent with the theoretical analysis. For example, with M = f ∗ n = 4096 , n = 2 f , we have the theoretical minimum f min = √ 2048 ≃ 45 (see Section 3.2). We can observe that the baseline Penguin (32 , 128) with f = 32 , n = 128 is very close to the theoretical minimum and achieves the best results among the three designs. Meanwhile, other HOCs besides Rotation may increase under the optimal packing and affect the overall latency. For example, Penguin (32 , 128) yields more ciphertext multiplication (CMult) than Penguin (16 , 256) due to wasted slots from feature reduction, which can be further optimized using the proposed Interleaved Assembling method.

Moreover, as our discussion of large graph in Section 3.2 indicates, all designs exhibit significantly worse performance in the PubMed dataset compared to the other two datasets. This is because the number of nodes in PubMed is significantly larger than the size of ciphertext (19717 ≫ 4096), which means that it needs to be multiplied with a large 19717 × 19717 adjacency matrix. Therefore, the number of CMult ≫ number of Rot. However, our proposed packing technique can still improve the performance in such cases.

4.2.2 Interleaved Assembling

Table 2 also reports the evaluation results of incorporating the two-dimensional parallel packing and interleaved assembly methods. For example, in Cora, the number of rotations, the number of CMult, and the number of other HOCs in the Penguin (32 , 128) + IA design are further reduced by 1.4K, 114.7K, and 31K, respectively, compared to the parallel-packing only Penguin (32 , 128) . This makes Penguin (32 , 128) + IA the best design on all datasets, i.e., with the minimum memory

usage of 0.49GB, 1.39GB, and 3.76GB and a 10 . 62 × , 9 . 96 × , and 1 . 46 × speedup on dataset Cora, Citeseer, and PubMed, respectively. These results illustrate that our proposed interleaved assembly can effectively reduce the wasted empty slots and save the number of ciphertexts selected in the computation, thus significantly improving the efficiency based on the SIMD.

4.2.3 Compare with State-of-the-art Solutions

In our evaluation, we conduct a comparative analysis of our best-designed model, Penguin (32 , 128)+ IA with several state-of-the-art (SOTA) solutions, including Gazelle [17], HElayers [1], E2DM [15] and uSCORE [13]. All these SOTA solutions can speed up HE-GCN inference with the optimized matrix-matrix multiplication. The results are summarized in Table 3.

It is worth noting that our approach uses encryption parameters which offer a 128-bit security level. This security level is higher than that of the other two SOTA solutions, which can only guarantee a 98-bit security level and require more multiplicative levels to mask the plaintexts. For a fair comparison, we measure the amortized latency-the time required for link predictions of a single node. As reported in Table 3, our method achieves an amortized latency of 0.24s on Cora, which is 5 . 92 × , 4 . 83 × , 3 . 25 × and 2 . 67 × faster than that of Gazelle, E2DM, HElayers and uSCORE, respectively. The similar improvement can be observed across the Citeseer and PubMed.

The superiority of our method over other SOTA solutions can be attributed to two key factors:

The first reason is that all prior approaches are primarily designed to solve the single matrix-matrix multiplication problem instead of two-way matrix multiplications in GCN inference. In GCN inference, the computation pattern mainly consists of two computational components-the common FC layer for graph feature aggregation ( X · W ) and the adjacency matrix multiplication for graph node aggregation ( A · X ). These two components represent distinct and orthogonal computation patterns for the input feature map. Consequently, optimizing only one of these directions leads to sub-optimal performance. In contrast, our proposed "Penguin", which incorporates two-way parallel encoding and an optimized sub-block matrix size, offers superior performance with theoretical guarantees.

The second compelling factor contributing to the excellence of our approach is the adaptability to address the issue of wasted slots of ciphertext, a capability notably absent in prior methods. In studies such as [15, 1, 13], traditional matrix-multiplication methods establish sub-block matrix partitions based on square or rectangular sub-block matrices determined by the input weight matrix size. However, these conventional methods face challenges in effectively managing the problem of wasted slots that arise after feature aggregation in Graph Convolutional Network (GCN) inference. As confirmed in Table 3, our two-way parallel packing method substantially improves HE-GCN inference by leveraging the Interleaved Assembling technique that can effectively minimize the wasted slots. This, in turn, leads to extra memory space and HOC reduction.

Putting all these together, our method clearly demonstrates its advantage over SOTA solutions in accelerating HE-GCN inference.

5 Related Work

CryptoNets [10] is the first work that demonstrates the feasibility of building privacy-preserving machine learning (PPML) by HE. However, the long inference latency and the inflexible packing format make it hard to be applied to large-scale models and datasets. Another following work named SHE [25], translates the nonlinear ReLU and Max Pooling operations as Boolean operations to support the TFHE-based [7] PPML without modifying the pretrained models. There also exist many multi-party computation (MPC) solutions that combine the two-party computation protocols [36] with HE frameworks to achieve the low inference latency [30, 17, 26, 24, 27]. However, they suffer from high communication overhead incurred by data transfer between multiple parties. Recent studies such as LoLa [4], CHET [8], and HEAR [20] leverage the ciphertext packing technique to place multiple data in the same ciphertext so that HE operations can be conducted efficiently via single instruction multiple data (SIMD) for accelerating HE-based CNN inference. These approaches are often not applicable or optimal to GCN inference due to the very different computation patterns between the GCN and CNN. CryptoGCN [28] is the first attempt to build HE-based PPML for GCNs. It packs the ciphertexts from individual node to relieve the adjacency matrix multiplication overhead. However, they assume the adjacency matrix as plaintext, which is not applicable to dynamic graph settings which require protecting both graph structure and features like our work.

Cheetah [14] aims to eliminate the need of costly rotation in HE-matrix multiplication. The basic idea is to replace the dot-sum operation with two polynomials multiplications. However, it requires communication between the client and server to perform ciphertext decryption and re-encryption after each layer computation, e.g. after a convolutional layer. In contrast, our approach operates within a 'HE without-client-aid" setting. Here, the server only requires the client to encrypt and send data to the server once, after which the server can perform the whole computation of inference and only send the final encrypted result back to the client. It does not require the frequent communication between client and server like [14]. Therefore, our work focuses on HE computation optimization. In our view, our solution and [14]'s MPC+HE setting represent two distinct directions to realize PPML. Each approach is tailored for different private inference scenarios. Hence, conducting a direct comparison between our method and the approach in [14] could be challenging since they are designed for different purposes and provide distinct advantages.

Gazelle [17] is a hybrid matrix multiplication algorithm based on mixture of naive implementation and diagonal encoding method [12]. It is designed to handle the general matrix multiplication that occurs in FC layers, where the output dimension is smaller than the input dimension. Additionally, E2DM [15] and uSCORE [13] address encrypted matrix-matrix multiplication optimization by breaking down the problem into small square matrix multiplications and facilitating consecutive matrix multiplications.HElayers [1], on the other hand, leverages a tiles-tensor-based matrix-multiplication technique for multiplying two square matrices and consumes less multiplicative depth compared to [15]. While these general solutions have proven effective in various contexts, their efficiency is limited when applied to accelerate Homomorphic Encryption-based Graph Convolutional Network (HE-GCN) inference, as detailed in Section 4.2.

6 Conclusion

In this paper, we propose a two-dimension parallel packing technique for feature ciphertext by optimizing the feature matrix partition size and further propose an interleaved assembling technique to merge ciphertexts that have wasted slots from feature reduction in CKKS-based secure GCN inference. These techniques can better save ciphertext memory and effectively reduce the number of homomorphic operations required. Experimental results based on the GAEs for link prediction and 3 popular graph datasets show that our solution can speed up the latency of the secure GCN inference by 10 × and reduce the memory requirement by more than 79%, greatly outperforming the state-of-the-art solutions.

7 Acknowledgment

We thank all anonymous reviewers for their constructive comments and suggestions on this work. This work is partially supported by the National Science Foundation (NSF) under Grants No. CNS2348733 and CNS- 2349538.

References

Ehud Aharoni, Allon Adir, Moran Baruch, Nir Drucker, Gilad Ezov, Ariel Farkash, Lev Greenberg, Ramy Masalha, Guy Moshkowich, Dov Murik, et al. Helayers: A tile tensors framework for large neural networks on encrypted data. Proceedings on Privacy Enhancing Technologies , 1(1):325-342, 2023.

Fabian Boemer, Anamaria Costache, Rosario Cammarota, and Casimir Wierzynski. ngraph-he2: A high-throughput framework for neural network inference on encrypted data. In Proceedings of the 7th ACM Workshop on Encrypted Computing & Applied Homomorphic Cryptography , pages 45-56, 2019.

Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. (leveled) fully homomorphic encryption without bootstrapping. ACM Transactions on Computation Theory (TOCT) , 6(3):136, 2014.

Alon Brutzkus, Ran Gilad-Bachrach, and Oren Elisha. Low latency privacy preserving inference. In International Conference on Machine Learning , pages 812-821. PMLR, 2019.

Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, and Yongsoo Song. A full rns variant of approximate homomorphic encryption. In International Conference on Selected Areas in Cryptography , pages 347-368. Springer, 2018.

Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. Homomorphic encryption for arithmetic of approximate numbers. In International Conference on the Theory and Application of Cryptology and Information Security , pages 409-437. Springer, 2017.

Ilaria Chillotti, Nicolas Gama, Mariya Georgieva, and Malika Izabachène. Tfhe: fast fully homomorphic encryption over the torus. Journal of Cryptology , 33(1):34-91, 2020.

Roshan Dathathri, Olli Saarikivi, Hao Chen, Kim Laine, Kristin Lauter, Saeed Maleki, Madanlal Musuvathi, and Todd Mytkowicz. Chet: an optimizing compiler for fully-homomorphic neuralnetwork inferencing. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation , pages 142-156, 2019.

Craig Gentry. A fully homomorphic encryption scheme . Stanford university, 2009.

Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In International conference on machine learning , pages 201-210. PMLR, 2016.

C Lee Giles, Kurt D Bollacker, and Steve Lawrence. Citeseer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libraries , pages 89-98, 1998.

Shai Halevi and Victor Shoup. Algorithms in helib. In Annual Cryptology Conference , pages 554-571. Springer, 2014.

Zhicong Huang, Cheng Hong, Wen-jie Lu, Chenkai Weng, and Hunter Qu. More efficient secure matrix multiplication for unbalanced recommender systems. IEEE Transactions on Dependable and Secure Computing , 2021.

Zhicong Huang, Wen-jie Lu, Cheng Hong, and Jiansheng Ding. Cheetah: Lean and fast secure two-party deep neural network inference. IACR Cryptol. ePrint Arch. , 2022:207, 2022.

Xiaoqian Jiang, Miran Kim, Kristin Lauter, and Yongsoo Song. Secure outsourced matrix computation and application to neural networks. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security , pages 1209-1222, 2018.

Wonkyung Jung, Eojin Lee, Sangpyo Kim, Jongmin Kim, Namhoon Kim, Keewoo Lee, Chohong Min, Jung Hee Cheon, and Jung Ho Ahn. Accelerating fully homomorphic encryption through architecture-centric analysis and optimization. IEEE Access , 9:98772-98789, 2021.

Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. { GAZELLE } : A low latency framework for secure neural network inference. In 27th USENIX Security Symposium (USENIX Security 18) , pages 1651-1669, 2018.

Oussema Keskes and Rita Noumeir. Vision-based fall detection using st-gcn. IEEE Access , 9:28224-28236, 2021.

Miran Kim, Xiaoqian Jiang, Kristin Lauter, Elkhan Ismayilzada, and Shayan Shams. Hear: Human action recognition via neural networks on homomorphically encrypted data. preprint arXiv:2104.09164 , 2021.

Miran Kim, Xiaoqian Jiang, Kristin Lauter, Elkhan Ismayilzada, and Shayan Shams. Secure human action recognition by encrypted neural network inference. Nature communications , 13(1):1-13, 2022.

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 , 2016.

Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308 , 2016.

Eunsang Lee, Joon-Woo Lee, Junghyun Lee, Young-Sik Kim, Yongjune Kim, Jong-Seon No, and Woosuk Choi. Low-complexity deep convolutional neural networks on fully homomorphic encryption using multiplexed parallel convolutions. In International Conference on Machine Learning , pages 12403-12422. PMLR, 2022.

Jian Liu, Mika Juuti, Yao Lu, and Nadarajah Asokan. Oblivious neural network predictions via minionn transformations. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security , pages 619-631, 2017.

Qian Lou and Lei Jiang. She: A fast and accurate deep neural network for encrypted data. Advances in Neural Information Processing Systems , 32, 2019.

Pratyush Mishra, Ryan Lehmkuhl, Akshayaram Srinivasan, Wenting Zheng, and Raluca Ada Popa. Delphi: A cryptographic inference service for neural networks. In 29th USENIX Security Symposium (USENIX Security 20) , pages 2505-2522, 2020.

Payman Mohassel and Yupeng Zhang. Secureml: A system for scalable privacy-preserving machine learning. In 2017 IEEE symposium on security and privacy (SP) , pages 19-38. IEEE, 2017.

Ran Ran, Wei Wang, Quan Gang, Jieming Yin, Nuo Xu, and Wujie Wen. CryptoGCN: Fast and scalable homomorphically encrypted graph convolutional network inference. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Ronald L Rivest, Len Adleman, Michael L Dertouzos, et al. On data banks and privacy homomorphisms. Foundations of secure computation , 4(11):169-180, 1978.

Bita Darvish Rouhani, M Sadegh Riazi, and Farinaz Koushanfar. Deepsecure: Scalable provablysecure deep learning. In Proceedings of the 55th annual design automation conference , pages 1-6, 2018.

Microsoft SEAL (release 3.7). https://github.com/Microsoft/SEAL , September 2021. Microsoft Research, Redmond, WA.

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina EliassiRad. Collective classification in network data. AI magazine , 29(3):93-93, 2008.

Minjie Wang, Da Zheng, Zihao Ye, Quan Gan, Mufei Li, Xiang Song, Jinjing Zhou, Chao Ma, Lingfan Yu, Yu Gai, Tianjun Xiao, Tong He, George Karypis, Jinyang Li, and Zheng Zhang. Deep graph library: A graph-centric, highly-performant package for graph neural networks. preprint arXiv:1909.01315 , 2019.

Le Wu, Peijie Sun, Richang Hong, Yanjie Fu, Xiting Wang, and Meng Wang. Socialgcn: An efficient graph convolutional network based model for social recommendation. arXiv preprint arXiv:1811.02815 , 2018.

Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. Graph neural networks in recommender systems: a survey. ACM Computing Surveys (CSUR) , 2020.

Andrew Chi-Chih Yao. How to generate and exchange secrets. In 27th Annual Symposium on Foundations of Computer Science (sfcs 1986) , pages 162-167. IEEE, 1986.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTo alleviate the dramatic computation and memory overhead in HE-based GCN inference, this paper proposes a new HE-based ciphertext packing technique named Penguin. Penguin focuses on a sequence of matrix-matrix multiplications which is the bottleneck during private GCN inference. Thus, it employs an effective two-dimension parallel packing technique and an interleaved assembly technique to reduce the number of HE rotations while make use of the blank slots in polynomials. This paper also provides theoretical analysis and experimental validation to demonstrate the speedup achieved by Penguin in accelerating private GCN inference. Results show that Penguin can achieve up to ∼ 10× speedup and around ∼ 79% reduction in computational memory overhead, outperforming SOTA solutions.\n\nSTRENGTHS:\n1. Reasonable motivation. This paper first analyzes the latency breakdown in private GCN inference and finds that the bottleneck is a sequence of matrix-matrix multiplications $A\\cdot X\\cdot W$ due to a great deal of Rotations and CMults. Additionally, the inefficient packing method is the deeper reason. Therefore, this paper proposes several packing techniques to solve this issue.\n\n2. Direct and effective method. Based on the node-wise and feature-wise packing, this paper proposes Two-Dimension Parallel-Packing which selects tiling size to minimize the number of rotations. What's more, when feature size is reduced in aggregation, Interleaved Assembling can be used to make a denser packing. Experiments demonstrate the effectiveness of the two methods.\n\n3. Well written. The writing quality of the paper is excellent, with logical coherence.\n\nWEAKNESSES:\n1. Missing comparison with several highly related works. The node-wise and feature-wise packing methods described in Sec 3 are actually common in SIMD-based HE methods. Thus, I think this paper needs to have a comparison with SIMD-based work like Gazelle [1].\n\n[1] Juvekar, Chiraag, Vinod Vaikuntanathan, and Anantha Chandrakasan. \"{GAZELLE}: A low latency framework for secure neural network inference.\" 27th USENIX Security Symposium (USENIX Security 18). 2018.\n\n[2] Reagen, Brandon, et al. \"Cheetah: Optimizing and accelerating homomorphic encryption for private inference.\" 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 2021.\n\n[3] Hao, Meng, et al. \"Iron: Private inference on transformers.\" Advances in Neural Information Processing Systems 35 (2022): 15718-15731.\n\n2. There exists another type of method named coefficient encoding which has extremely high performance in matrix-matrix and matrix-vector multiplication[2,3]. So the SOTA methods claimed by the author are questionable.\n\n3. The baseline of the paper is not very strong. How does the proposed method compare to CryptoGCN and other works that directly optimize HE for GCN instead of CNN?\n\nQUESTIONS:\n1. How does the proposed method compare to the coefficient encoding-based method like Cheetah?\n\n2. How about the comparison with stronger baselines like CryptoGCN that optimizes HE for GCN?\n\n3. In Table 2, Penguin(32,128)+IA significantly reduces the number of CMults over 10 times, but the latency only reduces 20% compared to Penguin(32,128). Why is this situation happening?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The paper introduces Penguin, a novel HE-based ciphertext packing technique for accelerating GCN inference on encrypted graph data while ensuring data privacy. By exploiting the unique computation pattern of GCN layers, Penguin reduces computation and memory overhead associated with HE operations. The technique achieves significant speedup and reduction in computational memory overhead compared to state-of-the-art solutions. This work is the first to address the protection of both graph structure and features in accelerating HE-GCN inference on encrypted data.

STRENGTHS:
Introduces Penguin, a novel HE-based ciphertext packing technique for accelerating GCN inference on encrypted graph data.
Exploits the computation pattern of GCN layers to reduce computation and memory overhead associated with HE operations.
Provides theoretical analysis and experimental validation to demonstrate the speedup achieved by Penguin.
Achieves significant speedup and reduction in computational memory overhead compared to state-of-the-art solutions.
Addresses the protection of both graph structure and features in HE-GCN inference on encrypted data.

WEAKNESSES:
It would be beneficial to provide more detailed comparisons with existing approaches to highlight the advantages of Penguin.

QUESTIONS:
-

LIMITATIONS:
The paper could benefit from further discussion on the limitations and potential future directions of the proposed technique.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed an efficient data-packing method for cryptographically-secure inference on GCN, where the feature matrix and adjacency matrix are encrypted using homomorphic encryption (HE). The problem statement is interesting as the GCNs typically exhibit a significant sparsity level, increasing the number of rotations and, consequently, causing an increase in the HE latency.\n\nSTRENGTHS:\n1. The proposed solution is *novel* for dealing with rotation inefficiency in HE, which stems from the high degree of sparsity in GCN. \n\n\n2. The presentation of the proposed solution is excellent, and the paper provides clear and detailed information about the experimental methodology, including the specific parameters of HE.\n\nWEAKNESSES:\n**Comparison with SOTA data packing method**\nTable 3 in the paper does not compare the proposed solution and the state-of-the-art HE-packing method [1]. It is worth noting that the HELayer [1] incorporates a packing optimizer that selects the most efficient packing method to optimize latency and memory usage in HE convolution operations. Conducting a study on the suitability of the HELayer packing for graph convolution and their comparison with the proposed solution can provide valuable insights.\n\n**Lack of experiments for showing the impact of sparsity on the efficacy of the proposed packing method**\nAn experimental study and discussion on the impact of sparsity on the effectiveness of existing/state-of-the-art HE packing methods versus the proposed methods in GCNs would provide valuable insights. \n\n\n**CryptoGCN + Penguin** \nIncluding an experimental analysis in the paper would have been beneficial to understand how the proposed approach in CryptoGCN (NeurIPS'22) for reducing the multiplicative depth can further enhance the efficiency of the proposed HE packing method.\n\n\n1. Aharoni et al., \"HElayers: A tile tensors framework for large neural networks on encrypted data, \" PoPETs 2023.\n\nQUESTIONS:\nCan we employ the rotation-free HE implementation [1] for faster private inference on GCNs? How does the sparsity in GCN impact the solution proposed for making HE rotation free [1]?  \n\n\n\n\n1.  Huang et al., \"Cheetah: Lean and fast secure two-party deep neural network inference,\" USENIX Security 2022.\n\nLIMITATIONS:\nNot addressed in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

comments: SUMMARY:
The paper proposes a framework for optimizing latency in secure inference of GNNs. The authors propose a CKKS based packing scheme that is tailored for the structure of operations applied in Graph Convolution Networks. This appears to be the first work that considers both adjacency matrix and the feature matrix as private.

STRENGTHS:
1. The optimizations for parallel packing introduced in this work are well motivated and justified with experimental data.
2. The technique provides a significant latency improvement over existing matrix multiplication techniques empirically.
3. I found the overall in-depth analysis of GCN computation and utilizing it for optimizing packing and reducing wasted operations with interleaved assembling to be well designed.

WEAKNESSES:
1. I wonder if a fair evaluation would include cryptoGCNs with encrypted adjacency matrix.
2. The latency is extremely high for practical purposes, however, there isn't much work in this specific problem therefore it is difficult to understand the difficulty of the task.
3. How does this technique compare with SecGNN[1]? (Ignoring the training aspect)

References -
1. Wang, Songlei, Yifeng Zheng, and Xiaohua Jia. "SecGNN: Privacy-preserving graph neural network training and inference as a cloud service." IEEE Transactions on Services Computing (2023).

QUESTIONS:
None

LIMITATIONS:
I could not find any discussion on limitations by the authors even though they claim it in the submission form.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-10