$\rm A^2Q$: Aggregation-Aware Quantization for Graph Neural Networks
Authors: Zeyu Zhu
Fanrong Li
Zitao Mo
Qinghao Hu
Gang Li
Zejian Liu
Xiaoyao Liang
Jian Cheng
Venue: ICLR 2023 poster (2023)
Keywords: Graph Neural Networks
MPNN framework
Mixed-precision
Quantization

Abstract: As graph data size increases, the vast latency and memory consumption during inference pose a significant challenge to the real-world deployment of Graph Neural Networks (GNNs). While quantization is a powerful approach to reducing GNNs complexity, most previous works on GNNs quantization fail to exploit the unique characteristics of GNNs, suffering from severe accuracy degradation. Through an in-depth analysis of the topology of GNNs, we observe that the topology of the graph leads to significant differences between nodes, and most of the nodes in a graph appear to have a small aggregation value. Motivated by this, in this paper, we propose the Aggregation-Aware mixed-precision Quantization ($\rm A^2Q$) for GNNs, where an appropriate bitwidth is automatically learned and assigned to each node in the graph. To mitigate the vanishing gradient problem caused by sparse connections between nodes, we propose a Local Gradient method to serve the quantization error of the node features as the supervision during training. We also develop a Nearest Neighbor Strategy to deal with the generalization on unseen graphs. Extensive experiments on eight public node-level and graph-level datasets demonstrate the generality and robustness of our proposed method. Compared to the FP32 models, our method can achieve up to $18.8\times$ (i.e., 1.70bits) compression ratio with negligible accuracy degradation. Moreover, compared to the state-of-the-art quantization method, our method can achieve up to $11.4\%$ and $9.5\%$ accuracy improvements on the node-level and graph-level tasks, respectively, and up to $2\times$ speedup on a dedicated hardware accelerator.

Published as a conference paper at ICLR 2023
QQQ: AGGREGATION-AWARE QUANTIZATION FOR GRAPH NEURAL NETWORKS
Zeyu 2. Fanrong Li
Zitao Mo!
Qinghao Hu²
Gang Li
Zejian Liu
Xiaoyao Liang'
Jian Cheng1.233+
Institute of Automation, Chinese Academy of Sciences
Schooo of Future Technology, University of Chinese Academy of Sciences
3 "AiRiA
*Shanghai Jiao Tong University
(zhuzeyu2021, 1ifanrong2017, mozitac2117eiia.aco nhuqinghao22144, liuze ac.la.acc.. en, {qliaca]esjtu. edu. cn, Liiang-x/ecs.sspp cha.en, en,
ABSTRACT
As graph data size increases, the vast latency and memory consumption during in- ference pose significant challenge the real-world deployment of Graph Neural Networks (GNNs). While quantization is a powerful approach to reducing GNNs complexity, most previous works on GNNs quantization fail exploit the unique characteristics of GNNs, suffering from severe accuracy degradation. Through an in-depth analysis of the topology of GNNs, we observe that the topology of the graph leads to significant differences between nodes, and most of the nodes in graph appear have small aggregation value. Motivated by this, in this paper, we propose the Aggregation-Aware mixed- preciipre Quantization (A (A²Q) for GNNs, where an appropriate bitwidth is automatically learned and assigned each node in the graph. To mitigate the vanishing gradient problem caused by sparse connections between nodes, we propose a Local Gradient method to serve the quantization error of the node features the supervision during training. We also develop a Nearest Neighbor Strategy to deal with the generalization on unseen graphs. Extensive experiments eight public node-level and graph-level datasets demonstrate the generality and robustness of our proposed method. Compared to the FP32 models, our method can achieve up to 18.6x (i.e., 1.70bit) compression ratio with negligible accuracy degradation. Morever, compared to the state-of-the- art quantization method, our method can achieve up to 11.4% and 9.5% accuracy improvements on the node-level and graph-level tasks, respectively, and up to 2x speedup on a dedicated hardware accelerator.
INTRODUCTION
Recently, Graph Neural Networks (GNNs) have attracted much attention due to their superior learn- ing and representing ability for non non-Euclidean geometric data. A number of GNNs have been widely used in real-world applications, such as recommendation system (Jin et al., 2020), and social net- work analysis (Lerer et al.) 2019), etc. Many of these tasks put forward high requirements for low-latency inference. However, the real-world graphs are often extremely large and irregular, such Reddit with 232,965 nodes, which needs 19G floating-pon point operations (FLOPs) to be processed by a 2-layer Graph Convolutional Network (GCN) with only 81KB parameters (Tailor et al.) 2020). while ResNet-50, 50-layer DNN, only takes 8G FLOPs to process an image (Canziani et al. 2016). What is worse, it requires a huge amount of memory access for GNNs inference, e.g., the nodes features size of Reddit is up 534MB, leading high latency. Therefore, the aforementioned problems pose a challenge to realize efficient inference of GNNs.
Neural network quantization can reduce the model size and accelerate inference without modifying the model architecture, which has become a promising method to solve this problem in recent years.
Conresponding author

---

Published as a conference paper at ICLR 2023
GCN
GIN Layer2
Avenge Fanurel (ffict A Aggnrioo)
GIN GAT
Avorage Ffatrre/Affe Aggntion)
GIN Layard
GIN Layers
60
[.000
121,000
[1500
PULL In dogree
[iskjax
In In-dogrec
(a)
(b)
Figure 1: The analysis of the average aggerated node features in different in-degrees node groups on various tasks. (a) The values at the final layer for GNNs trained on Cora. (b) The values at the 2-5 layer of GIN trained on REDDIT-BINARY. The average values are all generated from runs.
Unfortunately, there remain some issues in the existing works on GNNs quantization. Feng et al. (2020) only quantizes the node feature and keeps floating point calculations during inference. Tailor al. (2020) proposes degree-quant training strategy to quantize GNNs to the low-bit fixed point but causes large accuracy drop, e.g., 11.1% accuracy drops when quantizing to 4bits. Moreover, some works (Wang al. Ib Bahri al. 2021 Wang et al. 202 Jing et al., 2021) quantize GNNs into 1-bit and compute with XNOR and bit count operations. However, these 1-bit quantiza- tion methods are either restricted to the node-level tasks can not generalize well other GNNs.
Most of the above methods do not make full use of the property of GNNs and graph data, resulting in severe accuracy degradation poor generalization. As presented in MPNN framework (Gilmer et al. 2017). GNNs processing is divided into two phase: First, in the aggregation phase, a node collects information from neighboring nodes and uses the aggregation function generate hidden features; second, in the update phase, the hidden features are transformed into new features by an update function. We analyze the nodes features after aggregation Figure I and find that the higher the in-degree is, the larger the node features tend to be after aggregation. And the features vary significantly between nodes with different in-degrees. which represent the topology of graph. Moreover, according to Xie et al. (2014); Aiiell et al. (200T), the degrees of nodes in most real- world graph data often follow the power-law distribution, i.e., nodes with low degree account for the majority of graph data. Therefore, specially quantizing the nodes features according to the topology of the graphs will beneficial to reduce the quantization error while achieving higher compression ratio.
In this paper, we propose the Aggregation-Aware Quantization (A?Q) method, which quantizes different nodes features with different learnable quantization parameters, including bitwidth and step size. These parameters can be adaptively learned during training and are constrained by a penalty memory size improve the compression ratio. However, when quantizing the model in semi-supervised tasks, the gradients for most quantization parameters are zero due to the sparse connections between nodes, which makes the training non-trivial. We propose the Local Gradient method to solve this problem by introducing quantization error as supervised information. Finally, generalize our method unseen graphs in which the number of the nodes varies, we develop the Nearest Neighbor Strategy which assigns the learned quantization parameters to the unseen graph nodes. To the best of our knowledge, we are the first to introduce the mixed-pree -eexeedieeiisoonn quantization to the GNNs. Compared with the previous works, our proposed methods can significantly compress GNNs with negligible accuracy drop.
In summary, the key contributions of this paper are as follows:
1) We propose the Aggregation -awaee Aware mixed -recisiin Quantization (A (A?Q) method to enable adaptive learning of quantization parameters. Our learning method is powerful by fully utilizing the characteristic of GNNs, and the learned bitwidth strongly related to the topology of the graph.
2

---

Published as a conference paper at ICLR 2023
2) A Local Gradient method is proposed train the quantization parameters in semi- supervised learning tasks. Furthermore, to generalize our method to the unseen graphs in which the number of input nodes is variable, we develop the Nearest Neighbor Strategy to select quantization parameters for the nodes of the unseen graphs.
3) Experiments demonstrate that we can achieve a compression ratio up to 18.6x with negli- gible accuracy degradation compared to the full-precision (FP32) models. Moreover, the model trained with our A method outperforms the state-of-the-ar (SOTA) method up 11.4% with a speedup up to 2.00x in supervise tasks, and obtains up to 9.5% gains with a 1.16x speedup in graph-level tasks. We provide our code at this URL: https://github.com// wwwoo//wwwaiii///.. 98/AA
2 RELATED WORK
Graph Neural Networks: The concept of the graph neural network was first proposed in Scarselli eet al. (2008), which attempted to generalize neural networks to model non-Euclidean data. In the following years, various GNN models were proposed. For example, Graph Convolution Network (GCN) (Kipl & Welling 2016) uses a layer-wise propagation rule that is based on a first-order approximation of spectral convolutions on graphs, Graph Isomorphism Network (GIN) (Xu et al.] 2018) designed a provably maximally powerful GNN under the MPNN framework, and Graph At- tention Network (GAT) (Veličković et al. 2017) introduces the attention mechanism to graph pro- cessing. Although GNNs have encouraging performance in wide range of domains (Jin et al.) 2020 Yang 2019), the huge amount of float-point operations and memory access process pose a challenge to efficient inference, which hinder the applications of GNNs.
Quantized GNNs: As promising method to reduce the model size and accelerate the inference process, quantization is also applied to GNNs, Some works quantize features and weights in GNNs to low bitwidths (Feng et al. 2020 Tailor et al., 2020) or even 1-bit (Wang et al. 2021b Bahri et al,, 2021 Wang al.] 202 Jing al. [2021). Le., use fixed-point numbers instead of floating-poii numbers for computation. But when the compression ratio is high (e.g.. <4bit), the performance degradation of these works is significant, and the generalization of 1-bit method is limited. There are also some works on vector quantization (VQ), which use the vectors in a codebook obtained during the training process instead of the original features (Ding et al.) [202] Huang et al. 2022). However, searching for vectors in the codebook is computationally complex.
Mixed-Precision Quantization: Based on the idea that different layers have different sensitivities to quantization, mixed preccisinn quantization is proposed in CNNs to quantize different layers to different bitwidths for better model compression. Early works (Wang et al., 2019 Lou et al. 2019) proposed reinforcement learning (RL. based methods to search bitwidth for different layers, but they often require large computational resources, which limits the exploration of the search space. Another important class of mnixed-precision method is the criteria- based method, they use the spe- cific criteria to represent the quantization sensitivity, c.g., (Dong et al. 2019 2020 Chen et al.] quantiee different layers with different bitwidths based on the trace of the Hessian. Recently, there are some other methods to learn the bitwidth during training (Uhlich et al., 2019 Esser et al., 2019 Jain et al. 2020). However, due to the huge difference between GNNs and CNNs, is dif- ficult to use these methods on GNNs directly, and our A²Q is the first method to introduce the mixed-precision quantization to GNNs, further improving the inference efficiency of GNNs.
3 METHOD
In this section, we describe our proposed Aggregation-Aware Quantization in detail. Firstly, we present the formulation of the mixed-precision quantization for GNNs, which fully utilizes the prop- erty of GNNs and graph data. Secondly, introduce the Local Gradient method to address the gradient vanishing problem during training. Finally, we detail the Nearest Neighbor Strategy, which used for generalizing approach to the unseen graphs.
3

---

Published as a conference paper at ICLR 2023
Nedoo with lubel
Nodio witk ceeooee gradiees
Nodes
Pilt
F42
Figure 2: Perform matrix multiplication by the integer represented. Σ and t are both integers.
Figure 3: The gradients xy in GCN trained Cora by sampling 400 nodes.
3.1 AGGREGATION-AWALL QUANTIZATION
We assume a graph data with N nodes and the node features are a F'-dimensional, i.c., the feature map X E IR- and x; is the features of node i. We use the learnable parameters step size ag E R+ and bitwidth b; E R+ to quantize the features of the i-th node as:
2al a
+ 0.5J,,
|ac| a;:2bbb1-11
3; sign(x;))
2 20011-1 - 1.
|xy 2 a; (2(b:11-11
where I.] is the floor function, and [-] is the round function to ensure the bitwidth used quantize an integer. The learnable parameters are sx = (a1, ...,... a.), and bx = (b1, b2.... by)- Then we can obtain the fixed- feature map X, and the original feature can be represented as X, = Sx X, where Sx = diag(a) a2 ... n. Note that we use [b] + 1 as the quantization bitwidth for the features after ReLU because the values all non-negative.
In the update phase, the node features are often transformed with a linear mapping or an MLP in which matrix multiplication XW is the main computation, and the transformed node features the input to the next layer in GNNs. In order to accelerate the update phase, we also quantize W. Due to the fact that W in certain layer shared by all nodes, we quantize W the same bitwidth of 4bits for all GNNs in this paper. However, each column of W has its learnable quantization step size, i.e., sw (81. 32, Br,), where F2 the output-dimensio layer and 3 is the quantization step size for the i-th column of W, and we also use Eq. to quantize W. We can obtain the integer representation W and the quantized representation W,, W. Sw. where Sw = diag({3,, B2, .... }p,). The float-point matrix multiplication in the update phase can be reformulated as follow:
of the node features in current
W R X,, W, = (Sx (Sx-X) -W Sw) -X (X W) (sx X sw),
where O denotes an element-wise multiplication, and @ denotes the outer product. After training. we can obtain 8x and Sw so that the outer product can be pre-processed before inference. An example is illustrated in Figure 2 For the aggregation phase, i.c., AX, A is the adjacency matrix and A {0, N we quantize the X as the quantization way of W because the nodes features involved in the aggregation process come from the update phase, in which the features lose the topology information of graphs. Then the aggregation phase be performed by integer operations to reduce the computational overhead.
The quantization parameters (s, b) are trained by the backpropagation algorithm. Since the floor and round functions used in the quantization process are not differentiable, we use the straigh--throug estimator (Bengio al. 2013)) to approximate the gradient through these functions, and the gradients of the quantization parameters can be calculated by:
AL - 0s
OL dxi 0zt
ØL
d
dri
(3)
(4)
as
Zb
dir
ab
where d is the dimension of the vector æ. (s,b) are the quantization parameters for æ. and zi the value of i-th dimension in x,, Detailed information about quantization process and the backpropagation shown in Appendix AnI A.3 Proof 2 and 3.
4

---

Published as a conference paper at ICLR 2023
In order to improve the compression ratio of the node features, we introduce penalty term on the memory size:
Tymemory -
Σ [=1 i=1
dim'' (Maarget))
(5)
where I. is the number of layers in the GNNs, N is the total number of nodes, dim' is the length of the node features in 1-th layer, if the quantization bitwidth for node i 1-th layer, Maaggt the larget memory size the total node features memory size, and 17 8 1024, which is a constant to convert the unit of memory size to KB. Then the model and quantization parameters can be trained by the loss function:
Liotal = Lrask + X Lmemorr
(6)
where Leask is the task-related loss function and À is a penalty factor on Lmmeorr*
3.2 LOCAL GRADIENT
Although the above end-to-end learning method is concise and straightforward, the gradients for the quantization parameters of nodes features, i.e., 8haask and SLeank are almost zero during the 8s training process of semi-supervised tasks, which poses significant challenge to train the quantiza- tion parameters for nodes features. We analyze the property of GNNs and graph data, and find that two reasons lead this phenomenon: 1. The extreme sparsity of the connections between nodes in graph data. 2. Only tiny fraction of nodes with labels are used for training in semi-supervised tasks (e.g., 0.30% in PubMed dataset). Therefore, 0Ltazk for most node features are zero (detailed 0xx, proof in Appendix (A.3.2). which results in that the gradients for quantization parameters of these nodes vanish according to Eq. and Eq. 4 To clarify, we visualize the 8Lrakk in the second layer 00g of GCN trained on Cora. As shown in Figure 3 most gradients for the nodes features are zero.
äb
The gradients of the Lrask w.r.t. quantized nodes features can be viewed as the supervised infor- mation from the labeled nodes which enable the training of the quantization parameters for nodes features. However, this supervised information is missing due to zero gradients. Considering the quantization error is related to the Track we introduce the quantization error E 1. |ac, æI, as the supervised information for the quantization parameters of nodes features, where x: is the features before quantization, 2 is the features after quantization and - denotes the L1 norm. We refer to this method as Local Gradient because the gradients are computed by the local quantization er- rors instead of back proppprrpe taskerelated gradients. Then the quantization parameters for node features can be trained by gradients from E:
0F
Ox
OF
1
Ozj
sign( x²)
(7)
sign -x²)-
(8)
Ös
Ös
0b
Øb
Note that the quantization parameters of W are still trained by utilizing the gradients in Eq.
3.3 NEAREST NEIGHBOR STRATEGY
In graph-level tasks, the quantized GNNs are required generalize unseen graphs In such a scenario, the number of input nodes may vary during training or inference. However, the learnable method can only train fixed number of (s, b) pairs which are the same as the number of input nodes, so it is challenging to learn the s and b for every node in graph-level tasks. To solve this problem, we propose the Nearest Neighbor Strategy, which allows learning of a fixed number of quantization parameters and select quantization parameters for the unseen graphs
The proposed strategy shown Algorithm T To ensure the numerical range of Ty is as close at FP32, simple way is to keep the maximum quantization value equal to the maximum absolute value of æ. Based on this idea, we first initialize m groups of quantization parameters, then we calculate the maximum quantization value for every group, i.e., qmaz = s(2lb)- -1 -1). When quantizing the features of node i, the feature with the largest absolute value f. in the node features I; is first selected, and then we find the nearest qmaz and quantize the node features with the (s, b) corresponding to this Jmax- When performing backpropagation, we first calculate the gradients of the loss function w.r.t. quantization parameters according to Eq. B and Eq. 4 For a specific set of quantization parameters (8j, by), we collect the gradients from the nodes that have used them
5

---

Published as a conference paper at ICLR 2023
Algorithm Nearest Neighbor Strategy
ForwardPass (X (I11, 22, N Initialize 8, b), E Rmxx RR before training
2:
3: Calculate Qmaz o (20-1
Calculate the maximum absolute value in the features of each node: fi max abs (
4:
5:
Search the index of quantization parameters for each node: inder, = arg min Iff gunax
6:
Quantize the i-th node features using ( Dinden;;)
7:
return X,
8:
end
Table 1: The results comparison on node-level tasks.
The average bits are counted for each task
when the best results are achieved.
Dataset
Model
Accuracy
Average bits
Compression Ratio
Speedup
GCN(FP32)
81.520.7%
32
1x
GCN(DQ
1..3+1.7%
8x
1x 2.00x
GCN(ours)
80,9#0.6 %
1.70
18.6x
GIN(FP32)
77.6#1.1%
32
1x
Cora
GIN(DQ)
69.9+3.4%
4
8x
1x 1.41x
GIN(ours)
77.8+1.6 %
2.37
13.4x
GAT(FP32)
83. 83.1%0.4%
32
1x
GAT(DQ)
71.242.9 2.9
8x
1x 1.49x
GAT(ours)
82.620.6 %
2.03
15.4x
GCN(FP32)
71.1
32
1x
GCN(DQ
66.9+2.4%
8x
Ix 1.91x
GCN(ours)
70.6 %
1.87
17.0x
GIN(FP32)
60.1%
32
1x
CiteSeer
GIN(DQ)
60.8+2.19 1%
4
8x
1x 1.37x
GIN(ours)
65.1+1.7% %
2.54
12.6x
GAT(FP32)
72.3+0.7%
32
1x
GAT(DQ
67. 1.55%
8x
1x 1.45x
GAT(ours)
71.9 %
1.94
16.2x
GAT(FP32)
79.0=0.3%
32
1x
PubMed
GAT(DQ)
70.6 12.5%
8x
1x 1.38x
GAT(ours)
78.8+0.49 %
2.12
15.1x
GCN(FP32)
71.740.3%
32
ogbn-arxiv
GCN(DQ)
65.4+3.9%
4
8x
1x
GCN(ours)
71.1+0.3%
2.65
12.1x
1.28x
and add these gradients together. After the model has been trained, we obtain the quantization parameters (s, b). Since Omaz can be calculated and sorted in advance, searching the nearest Jmaz can be implemented by binary searching. Usually, we set 1000 for all graph-level tasks in our paper and the overhead introduced to inference time negligible.
EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
In this section, evaluate method on three typical GNN models, i.e., GCN, GIN, and GAT. And we compare our method with the FP32 GNN model and DQ-INT4 (Tailor et al., 2020) on eight datasets, including four node-level semi-learning tasks (Cora, CiteSeer, PubMed, ogbn-arxiv) (Hul et al. 2020 Yang et al. 2016) and four graph-level tasks (REDDIT-BINARY, MNIST, CIFARIO, ZINC) Yanardaag & Vishwanathan
2015 Dwivedi et al. 2020), to demonstrate the generality and

---

Published as a conference paper at ICLR 2023
Table 2: The results comparison on graph-level tasks.
Dataset
Model
Accuracy (Loss!)
Average bits
Compression ratio
Speedup
GCN(FP32)
90.1+0.2%
32
1x
GCN(DQ)
1..4+t.3%
1x 1.17x
GCN(ours)
89.9
3.50
9.12x
GIN(FP32)
96.4+0.4%
MNIST
GIN(DQ)
95.5+0.4%
4
8x
1x 1.07x
GIN(ours)
95.7+0.2%% %
3.75
8.52x
GAT(EP322)
95.6+0.1%
32
1x
GAT(DQ)
93.1+0.3%
8x
1x 1.13x
GoT(ours)
93.9=1.0% %
3.86
8.28x
GCN(FP32)
55.9+0.4%
32
1x
GCN(DQ)
51.1+0.7%
1x 1.25x
GCN(ours)
52.5+0.8%
3.32
9.62x
GIN(FP32)
57.5+t0.7%
CIFARIO
GIN(DQ)
50.7+1.6%
4
8x
1x 1.14x
GIN(ours)
54.941.5%
3.53
9.06x
GAT(EP32)
63.4+0.4%
32
Ix
GAT(DQ)
56.5+t0.6%
8x
1x 1.12x
GAT(ours)
64.7+2.89 %
3.73
8.57x
GCN(FP32)
0.450=0.008
32
1x
ZINC
GCN(DQ)
0.536#0.011
lx 1.08x
GCN(ours) 0.492=0.056
3.68
8.68x
GIN(FP32)
92.242.3%
1x
REDDIT- BINARY
GIN(D)) 41.3+4..4%
4
8x
1x 1.16x
GIN(ours)
90.8+1..99
3.50
9.14x
robustness of method. Among these datasets, ZINC is dataset for regression tasks, which uses regression loss as the metric of the model performance, while others are all for classification tasks.
For fair comparison, set the quantization bitwidth of W for all GNNs 4bits DQ-INT4. We count the average bitwidths for nodes features in all layers of the overall model and list them in our results, denoted by "Average bits". Since today's CPUs and GPUs can not support mixed-precisee operations well, we implement a precision-scalabe hardware accelerator to perform the overall in- ference process for GNN. The accelerator employs massive bit-serial multipliers Judd et al. (2016), therefore, the latency of the integer multiplications is determined by the bitwidth of the node Tea- tures. To evaluate the performance gains our method over DQ-INT4, we develop cycle-accurate simulator for our accelerator. More details about accelerator architecture are shown in Appendix A.7.5 Moreover, we show the compression ratio quantized GNNs compared to the FP32 models in terms of overall memory size. For simplicity, we use GNN(DQ) to represent the GNNs quantized by DQ-INT4 and GNN-dataset to represent the task in which we run the experiment, c.g., GCN- Cora represents the GCN model trained Cora. Detailed information about datasets and settings in Appendix A.5 and Appendix A.6
4.2 NODE-LAVEL. TASKS
Table shows the experimental results on three GNN architectures trained on four node-level datasets. Compared with DQ-INT4, our method can achieve significantly better accuracy on each task, even with a higher compression ratio, improving the inference performance with 1.28x to 2.00x speedups. On almost all node-level tasks, our proposed A A has negligible accuracy drop compared to the FP32 baselines while achieving 12. 12-1x-18.8 18.66 compression ratio. Since both GIN and GAT in- volve more complex computations, such the calculation of attention coefficients in GAT, it is more challenging to quantize those models, and DQ performs poorly on these two models. How- method can overcome this problem and maintain comparable accuracy compared with the FP32 models. Our method can outperform the DQ-INT4 by 11.4% on the GAT-Cora task with a smaller bitwidth (2.03 4). Even ogbn-arxiv, which has large number of nodes, A 2 can

---

Published as a conference paper at ICLR 2023
69-
5k
8k
3349
2.0-
400 150
21.
[k
2208
0.5-
10c
50
Rit
R0
198t
Rii
15.2
(a) GCN-CiteSeer
(b) GIN-CiteSeer
(c) GAT-CiteSeer
(d) The first layer (e) The second layer
Figure 4: The relationship between quantized bitwidth and average in-degrees of nodes. (a), (b) and (c) represent the results three GNN models trained on CiteSeer. (d) and (c) are results about the first and the second layer of MIP,, which is the update function of GIN trained REDDIT- BINARY.
achieve a 1x.1x compression ratio compared with FP32 baseline with comparable accuracy, which demonstrates the robustness our method. Moreover, to demonstrate the generality of our method, we also evaluate our method on heterogeneous graphs and the inductive learning tasks and compare with more related works in Appendix A.7.1
4.3 GRAPH-LEVEL TASKS
Table2present the comparison results the graph-level tasks. Our method can obtain better results on all tasks than DQ-INT4 with higher compression and a considerable speedup. Especially on the BEDDIT-BINARY task, method outperforms DQ-INT4 by 9.5%. Even for graph datasets with similar in-degrees, e.g., MNIST and CIFAR10, our method also learns the appropriate bitwidths for higher compression ratio and better accuracy. Although GIN-MINST task, the improvement of our method relatively small due to the similarity of the in-degrees between different nodes, our method can achieve comparable accuracy with smaller bitwidth (3.75 4).
4.4 ANALYSIS
To understand why our approach works, we analyze the relationship between the learned bitwidths and the topology of the graph. Figure 4(a) and 4(b) reveal that the bitwidth learned by A A is strongly related to the topology of graph data in the node-level tasks. As the bitwidth increases, the average in-degrees of nodes become larger. In other words, A²Q method tends to learn higher bitwidth for nodes with higher in-degrees. However, in GAT, as shown in Figure [(c the learned bits irregular. This is because the features aggregated in GAT our method can still learn appropriate quantization bitwidths for different nodes, which improves accuracy while reducing memory usage. In addition, Figure alsso shows the node distribution for different bitwidths and the result consistent with power-law distribution. Since nodes in graph data mainly have low in-degrees, most of the nodes are quantized to low bitwidth (S 4), compressing the GNNs as much as possible. And there are also some high in-degree nodes quantized to high bitwidth, which can help maintain the accuracy of the GNN models. As result, the average bitwidth of the entire graph features is low, and the accuracy degradation is negligible.
applogyyfreee. However,
For the graph-level tasks in which the number of nodes varies, our method also aggregation- aware. We select layer of GIN trained on REDDIT-BINARY and analyze the relationship between bitwidth and average in-degrees of nodes using the corresponding bitwidth to quantize Figure and 4(e) It can be seen that the bitwidth learned for nodes features input to the second layer of MLP, which is the update function in GIN for graph-level tasks, does not present correlation with the topology of graph. We analyze the reason and find that the node features before the second layer the result mapped by the first layer of MLP and activated by the activation function, e.g., ReLU, which results in the node features losing the topology information. We present more experiment results in Appendix A.7 to demonstrate that our method is generally applicable.
5 ABLATION STUDY
The advantage of learning-based Q with the manual mixeddprecisionn method, which manually assigns high-bit bit to those nodes with high in-degrees and low-bit to those nodes with low in-degrees. In the figure, the postfix "leam"
quantization: In Figure we compare
8

---

Published as a conference paper at ICLR 2023
Table 3: Ablation Study,
79.0%
Test Accuracy (%)
10
18.4%
Average bits
Model
Config
Accuracy
no-Ir
33.7:4.1%
1224 Coareen
30
000,00
no-Ir-b
75.6+0.2%%
4
133 Ei NT1
GIN-Cora
no-Ir-s
56. 1+4.9%
3.85
Ir-all
77.8+1.6% %
2.37
FP32
71.1=0.7%
32
Average Bits
GCN-
Global
56.8+6.798
CiteSeer
Local
70.6:11.1 %
1.87
Figure 5: The comparison between learning bitwidth and assign manually.
denotes that using A A method, "manual" denotes that we assign bits to nodes and the model only learns the stepsize, and "mixed- -reexeeirreisioon denotes that the model uses the same quantization method as DQ-INT4 but assigning different bitwidths nodes. For the "mixed srixxed-preisionn, we assign 5bits to those nodes with 50% top in-degrees and assign 3bits to others. 'he implications are two-fold. First, compared with the DQ-INT4, which uses the same quantization bitwidth, the mixed-precision method obtains 1.1% gains GCN-Cora tasks demonstrating that the mixed-precision ppeeciiioon method is more effective. Second, the results of the learning method outperform the manual method on all tasks. Especially for the models with high compression ratio, GIN-CiteSeer task, learning method can achieve 21.5 11.5% higher accuracy. This demonstrates that our learning method can perform better than the assignment method according prior knowledge for mixed-precision precision quantization of GNNs.
The power of learning the quantization parameters: Ablations of two quantization parameters (s, b) on the GIN-Cora task are reported in the first row of Table 3 The "no-Ir" denotes that do not use learning method, "no-Ir-b" denotes that only learn the step size s "no-Ir-s" denotes that only learn the bitwidths b, and "Ir-all" denotes that learn the bitwidth and step size simultaneously. We can see that learning the step size can significantly increase the accuracy and even the "no-Ir-bit" model can outperform the DQ-INT4 at the same compression ratio. When learning the bitwidth and step size simultaneously, the model can achieve higher accuracy with a higher compression ratio. This is because our method learns lower bitwidths for most nodes with low in-degrees and higher bitwidths for a tiny fraction of nodes with high in-degrees, which can improve the compression ratio while achieving higher accuracy.
Local Gradient v.s. Global Gradient: To demonstrate the effectiveness of our Local Gradient method, we compare the models trained with and without it on the GCN-CiteSeer task in the last row of Table 3 The "Global" denotes that the model is trained with Eq. 3 and Eq. 4 The model trained with the local method outperforms the global method by 13.8% with higher compression ratio. This is because the Local Gradient method can learn quantization parameters for all nodes, be updated with the Global Gradient
while only quantization parameters for part of nodes method due to the extreme sparse connection in the graph on the node-level semi-supervised tasks.
The overhead of Nearest Neighbor Strategy: We evaluate the real inference time of the GIN model on the 2080ti GPU. On REDDIT-BINARY task, the model without the selection process requires 121.45ms, while it takes 122.60ms for the model with our Nearest Neighbor Strategy. which only introduces 0.95% overhead. But with the help of the Nearest Neighbor Strategy, our model can obtain 19.3% accuracy gains for quantized GIN on REDDIT-BINARY.
6 CONCLUSION
This paper proposes A AQQ, an aggregation-aware mixed-precision quantization method for GNNs, and introduces the Local Gradient and Nearest Neighbor Strategy to generalize A²Q to the node- level and graph-level tasks, respectively. Our method can learn the quantization parameters for different nodes by fully utilizing the property of GNNs and graph data. The model quantized by our A2Q can achieve up to a 18.6x compression ratio, and the accuracy degradation is negligible compared with the FP32 baseline. Compared with the prior SOTA, DQ-INT4, significantly improve 11.4% accuracy with up to a 2.00x speedup on different tasks. Our work provides a general, robust and feasible solution to speed up the inference of GNNs.
method can

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work was supported in part by National Key Research and Development Program of China (No. 2020AAA0103400, the Strategic Priority Research Program of Chinese Academy of Sciences (No. XDB32050200), Jiangsu Key Research and Development Plan (No.BE2021012-2), and National Natural Science Foundation of China (No.62106267).
REFERENCES
Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süisstrunk. Slic superpixels compared to state-of-the-art superpixel methods. IEEE transactions on pattern analysis and machine intelligence, 34(11)2274-2282, 2012.
William Aiello, Fan Chung, and Linyuan Lu. A random graph model for power law graphs. Exper- imental mathematics, 10(1):53-66 66, 2001.
Mehdi Bahri, Gaétan Bahl, and Stefanos Zafeiriou. Binary graph neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2021.
Rajeev Balasubramonian, Andrew B Kahng, Naveen Muralimanohar, Ali Shafiee, and Vaishnav Srinivas. Cacti 7: New tools for interconnect exploration in innovative off-chip memories. ACM Transactions on Architecture and Code Optimization (TACO), 14(2):1-25, 2017.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: 3308.3432, 2013.
John Brennan, Stephen Bonner, Amir Sappour-Abarhouuse Philip T Jackson, Boguslaw Obara, and Andrew Stephen McGough. Not half bad: Exploring half precisrec in graph convolutional neural networks. In 2020 IEEE International Conference on Big Data (Big Data), pp. 2725-2734. IEEE, 2020.
Alfredo Canziani, Adam Paszke, and Eugenio Culurciello. An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07676 2016.
Weihan Chen, Peisong Wang, and Jian Cheng. Towards mixed-precision quantization of neural networks via constrained optimization. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 5350-5359, 2021.
Mucong Ding, Kezhi Kong, Jingling Li, Chen Zhu, John Dickerson, Furong Huang, and Tom Gold- stein. Vq-gmn: A universal framework to scale up graph neural networks using vector quantiza- tion. Advances in Neural Information Processing Systems, 34:6733-6746, 2021.
Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Hawq: Hessian aware quantization of neural networks with mixed- precision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 293-302, 2019.
Zhen Dong, Zhewei Yao, Daiyaan Arfeen, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Hawq-v2: Hessian aware tragh-weighted guantization of neural networks. Advances in neural information processing systems,
2020.
Vijay Prakash Dwivedi, Chaitanya Joshi, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks. arXiv preprint arXi::2003.00982, 2020.
Steven K Esser, Jeffrey McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmen- dra S Modha. Learned step size quantization. arXiv preprint arXivv1902.08753. 2019,
Boyuan Feng, Yuke Wang. Xu Li, Shu Yang. Xueqiao Peng. and Yufei Ding. Sgquant: Squeezing the last bit on graph neural networks with specialized quantization. In 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI). pp. 1044-1052. IEEE, 2020.
Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv: / 09099994322220 2019.
10

---

Published as a conference paper at ICLR 2023
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning. pp- 1263-1272. PMLR. 2017.
Rafael Gómez-Bombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Langelz--engeling, Dennis Sheberla, Jorge qquilera-Iparrauurre Timothy Hirzel, Ryan P Adams, and Alán Aspuru-Guzik. Guzik. Automatic chemical design using data-driven contin- uous representation of molecules. ACS central science, 4(2):268 276, 2018.
Will Hamilton, Zhitao Ying. and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A Horowitz, and William J Dally. Fie: Efficient inference engine on compressed deep neural network. ACM SIGARCH Computer Architecture News, 44(3):243-254, 2016.
Weihua Hu, Matthias Fey, Marinka Zitmik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in nearal information processing systems, 11222188-2111 2020.
Linyong Huang, Zhe Zhang, Zhaoyang Du, Shuangchen Li, Hongzhong Zheng, Yuan Xie, and Nianxiong Tan. Epquant: graph neural network compression approach based on product quan- tization. Neurocomputing, 503:49-61, 2022.
Sambhav Jain, Albert Gural, Michael Wu, and Chris Dick. Trained quantization thresholds for accurate and efficient fixed-point inference of deep neural networks. Proceedings of Machine Learning and Systems, 2:1 12-12-11 122. 2020.
Bowen Jin, Chen Gao, Xiangnan He, Depeng Jin, and Yong Li. Multi-behavior recommendation with graph convolutional networks. In Proceedings of the 43rd International ACM SIGIR Con- ference on Research and Development in Information Retrieval, pp. 659-668, 2020.
Yongcheng Jing, Yiding Yang, Xinchao Wang, Mingli Song, and Dacheng Tao. Meta-aggregator: Learning aggregate for 1-bit graph neural networks. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 5301-5310, 2021.
Patrick Judd, Jorge Albericio, Tayler Hetherington, Tor M Aamodt, and Andreas Moshovos. Stripes: Bit-serial deep neural network computing. In 2016 49th Annual IEEE/ACM International Sym- posium on Microarchitecture (MICRO), pp. 1-12. IEEE, 2016.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. arXiv preprint arXiv: (1.600009000 2016.
Adam Lerer, Ledell Wu, Jiajun Shen, Timothee Lacroix, Luca Wehrstedt, Abhijit Bose, and Alex Peysakhovich.
A large scale graph embedding system. Proceedings of Ma- chine Learning and Systems, 1: 121-131, 2019.
Qian Lou, Feng Guo, Lantao Liu, Minje Kim, and Lei Jiang, Autoq: Automated kernel-wise neural network quantization. arXiv preprint 1rXiv:1902.05690, 2019.
Mike O'Connor. Highlights of the band-hhnd memory (hbm) standard. In Memory forum workshop, volume 3, 2014.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini.
The graph neural network model. IEEE transactions on neural networks,
2008
Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. Efficient processing of deep neural networks. Synthesis Lectures on Computer Architecture, 15(2):1-341, 2020.
Shyam A Tailor, Javier Fernandez-Marques, and Nicholas D Lane. Degree-quant: Quantization- aware training for graph neural networks. arXiv preprint arXiv: 050000 2020.
Stefan Uhlich, Lukas Mauch, Fabien Cardinaux, Kazuki Yoshiyama, Javier Alonso Garcia, Stephen Tiedemann, Thomas Kemp, and Akira Nakamura. Mixed precision dnns: All you need is a good parametrization. arXiv preprint arXiv: 1452, 2019.

---

Published as a conference paper at ICLR 2023
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:/710.10003. 2017.
Hanchen Wang, Defu Lian, Ying Zhang, Lu Qin, Xiangjian He. Yiguang Lin, and Xuemin Lin. Binarized graph neural network. World Wide Web, 24(3):825-848, 2021a.
Junfu Wang, Yunhong Wang. Zhen Yang. Liang Yang. and Yuanfang Guo. Bi-gen: Binary graph convolutional network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1561-1570, 2021b.
Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han. Haq; Hardware-aware automated quan- tization with mixed precision. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8612-86222 8620, 2019.
Cong Xie, Ling Yan, Wu-Jun Li, and Zhihua Zhang. Distributed power-law graph computing: Theoretical and empirical analysis. Advances in neural information processing systems, 27, 2014.
Keyulu Xu, Weihua Hu, Jure Leskovee, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint 1rXiv:1810.000 2018.
Pinar Yanardag and SVN Vishwanathan. Deep graph kernels. In Proceedings the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1365-1374, 2015.
Hongxia Yang. Aligraph: A comprehensive graph neural network platform. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 3165- 3166, 2019.
Zhilin Yang. William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In International conference on machine learming, pp. 40-48. PMLR, 2016.
Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor Prasanna. Graph- saint: Graph sampling based inductive learning method. arXiv preprint D.XXv:/997.04922 2019.
Yiren Zhao, Duo Wang, Daniel Bates, Robert Mullins, Mateja Jamnik, and Pietro Lio. Learned low precision graph neural networks. arXiv preprint htt::20009092332 2020.
12

---

Published as a conference paper at ICLR 2023
A
APPENDIX
A.1 UNIFORM QUANTIZATION
In this section, we will give detailed introduction the content related quantification.
A.1.1
QUANTIZATION PROCESS
For a vector æ, the Ty is a quantized representation. Given the quantization step size s, 8 E R_,,, and the quantization bitwidth 6, b N4, then the uniform quantization is implemented as:
[2l + 0.5l.
[x]]
s(2
1)
i sign(x)
2h-1
x (2h-1 1)
The at 32bits is mapped to the integer number set {-26-1 + 1, .... 0,,., 2b bitwidth is #bb bits, and the quantized representation can be calculated as Bg = 8 S. For inference, can be used to compute matrix multiplication in the update phase perform other computations in GNNs layers and the output these computations then are rescaled by the corresponding s using relatively lower cost scalar-vecto multiplication. As illustrative example, for vectors and y E R the quantization parameters are both = 0.1, b 5, the process of inner product between these non-negative, we do not need to represent the sign bit in the fixed-poii representation. Therefore, the value can use #b bits to quantize instead of using the first bit to represent the sign bit. Then the quantization range of uniform quantization is [-s(2" 1)), s(2 1)].
1} where the
R3x1
vectors by integers is shown in Figure 6 When the values in vector are all
0.31
1,29
0.58
-0.436
-0.27
0.1
Preform by cloat-point representation
0.1
0..1
AigntO. 11)x
=1
0.1x 0 11551.5
0.1
-48
Rosale
-48
X 5,5,
-0.48
021
+0.
011
Quastization process
Perform by intagers representation
Figure 6: An example of performing inner product by integers representation.
A.1.2 GRADIENT IN BACKPROPAGATION
Due to the floor function used in the quantization process is not differentiable, the gradient of 2i with respect to æ vanishes almost everywhere, which makes it impossible to train the model by the backpropagation algorithm. Therefore, we use the straight-through estimator (Bengio ØL 3L
2013 to
approximate the gradient through the floor function, i.e., de arr, (1115-(2-1)) where [m|S>
a indicator function, whose value is 1 when |x| S s(2 1). and vice versa. In our paper, the
quantification parameters (s. b) are learnable, the gradients of xig w.r.t. (s, b) used in Eq. 3 and Eq.
arr::
(Eg xx)
Bay
ac s(20-1
He
(10)
(2 -1)
sig)(x))
xc > 5(20-1
35
20-1 In (2)
13

---

Published as a conference paper at ICLR 2023
Table 4: The aggregation functions and update functions for GNNs used in this paper, d, denotes the degree of node i, the denotes a learnable constant, and a represent attention coefficients.
Model
Aggregation function
Update function
GCN
h E
1111)
x 1 ReIUUWWn, + b())
Vdd /di
(EN(F)U())
GIN
h = (1 e (1-1)
+ Σ
11-1)
x = MLP W
(EN(s)
GAT
100 E
(1)
11-1)
x -w
JEN(UUUE)
Table 5: The statistics for density of adjacency matrix and the labeled nodes in four node-level datasets.
Cora
CiteSeer
PubMed
ogbn-arxiv
Density of 0444%
0.112%
0.028%
0.008%
Labled nodes
5.17%
3.61% %
0.30%
53.70%
A.2 MORE ABOUT GRAPH NEURAL NETWORKS
In this section, we first give detailed information about the MPNN framework (Gilmer et al. 2017). and then provide detailed examination of the three GNNs used in papers.
graph = EV, consist nodes V {l,... NN and edges E C x has node features X R NXF and optionally H-dimensional edge features E E RE The MPNN framework can (1-1) c.j ()-1)))) where (2 .
be formulated by a!!) (2 (z(1-1)
differentiable
(EN(1)
kernel function, the aggregation function which is permutation permutation-nnvariiatt and the y is learnable update function, at is the features of node and eij is the features of edge between node i and j. ((1)= (j (i,j) E E). and 1 represents the I-th layer of the GNNs.
In this paper, we focus three typical GNN models whose forwardpass all can be represented by the MPNN framework, Graph Convolution Network (GCN) (Kipf & Welling, 2016), Graph Isomor- phism Network (GIN) (Xu et al.) 2018), and Graph Attention Network (GAT) (Veličković et al. 2017). the detailed information shown in Table
A.3 PROOFS OF THEORETICAL RESULTS
This section provides formal proof of the theoretical results of our paper.
A.3.1 NOTATIONS
Here, we define the notations utilized in our proof. = indicates whether there is edge between each pair of nodes, e.g., if there is edge between node i and node then dg = otherwise, ai 0. Then, Ā A + 7 is the adjacency matrix for graph
= f0, N>NNNN is the adjacency matrix that
14

---

Published as a conference paper at ICLR 2023
that is added to the self-loops. The degree matrix D d'iag(d1, d2,..,,d)) dn), where d; =, aij and the degree matrix for the graph having self-loops is = (d], d2,..., da), where di = -, dij.
A.3.2 PROOFS
Proof 1. The gradients of the loss function with respect to the node features semi-supervised tasks are most zero.
Without loss of generality, we use the GCN model an example. From the Table 4 the graph convolution operation can be described as
x O (AX (XX"
(11)
where A D is the normalized adjacency matrix, W(() E Rf is a learnable weight matrix in the 1-th layer of GCN. ( is the input the (-th layer and the output of the (1 1)-th layer in GCN. a the non-linear activation function, e.g., ReLU. Note that the Á is an extreme sparse matrix for node-level datasets in our paper.
In our training process of the model, we use nll_ loss as our task loss function L. Only the nodes in the train set T have labels. For the last layer of GCN, we get the node features to be classified by H somtmar (1+1)). Then the gradient of I with respect ( is
OL
0H
G¹ =xxan (H(+T
[tyl E R NNFoue
(12)
0x (1+1)
where only the G i T is not zero, otherwise, G 0. Then, the gradient of the loss function with respect to x is
=²
(V L @ o' (Áx W (W
(13)
For node j do not have an edge with the node i, T. G 0. Table lists the density of the adjacency matrix A and the percentage of the labeled nodes in four nove- datasets. Because the sparsity property of adjacency matrix and the nodes with trained labels only account for tiny fraction of the graph, the gradients from the loss function for most node features are zero.
Proof 2. The normalized adjacency matrix Å is not needed to quantized for the GCN model.
We take the process of XW A(XW) an illustrative example, which represents first calculate B = XW and then calculate AB. For the (-th layer FP32 models, the first stage is B, W, and then calculate the X = AA,, where X, NxF; Wi E & X and A NxN The step-size for Bd, X, and Wi is Sa, Sx, and Sw., respectively. And they are all diagonal matrices. The integer representations are calculated as B = BLGSB. X Xx,X1g and W, = Ww, Note that for the node-level tasks, we obtain the SB, Sx, and Sw; in advance. And for the graph-level tasks, we can obtain them through one more element-wise multiplication whose overhead is negligible, as the comparison Table Then the first stage is:
B X W. (Sx, X1,) (Wi, Sw;) and there exists B: SB. Therefore, the integers representation for the next stage can be calculated as:
(14)
B1g = B.S
(Sx, Xzg) (WL. )5
(15)
(Sx, Xzg) (WL, (Sw, Sp,1
(Sx, @ (Sw, ( o (X1g Wt,)
where the (Sx, @ (Sw, (Sw.SF.) tion BLg for the next stage and do not introduce overhead.
S )) can be calculated offline. Then we obtain the fixed-point representa-
The process of node degree normalization after the aggregation process be represented as X oAAB), where Á D- ÄD- is the normalized adjacency matrix, and is the
15

---

Published as a conference paper at ICLR 2023
Offline
W
Quarnierr
W
NSS
X
Quantizrr
NNS
F Fused with the NNS from last stage
Fuse Execute
SX
Figure 7: The pipeline of the quantization process accelerator.
non-linear activation function. D- at the right side of can be fused into the Sx, and then calculate Bt.g as Eq.
Then the features the (L+ 1)-th layer X1+1 can obtained as X = o(D ABtg). And there exits X1+1 = Sx ((+1). Therefore, the X(+1).. can be obtained
X(+1), = =S
Xx41 X1+1
(16)
=S o o AB
Note that the elements in diagonal matrix Sx1+1 are all positive because this matrix is made up of step-size, which is always positive. Then we can obtain X(41)4 (Sx D A where Sx D can be obtained before inference and À E 10, The computation of AB only has addition operations and the x D can be obtained before inference for node-level tasks or introduce only once more element- wise multiplication calculate for the graph-level tasks.
The D the left side is fused into the element-wise multiplication performed by the next layer and the D at the right side is fused into the element-wise multiplication performed by the current layer and the element-wis. multiplication is necessary stage in the quantized model. Therefore, we can perform the node degree normalization using fixed point addition operation instead of quantizing the normalized adjacency matrix which may introduce
quantization error.
Proof 3. The quantization process can be fused with Batch Normalization operations.
When GNNs have Batch Normalization (BN) Layers, the calculation process is as follows (Note with the learned parameters BN):
that we have fused the mean and
X (N((((AB..)) = =(AB Y +
(17)
where = diag(y): y2....,FFF VF.)) E PaxE, Z = (x1 22, 22,... zF,) E NxF, and &; = (0i, 0c, ..., ..,...,,))" T E I. among which yt and 0, are the BN parameters for the i-th dimension fea- ture the nodes features. And there exits that X = Sx.., X(+1.. Therefore,
X S X X141
=S (o(ABI,) Y
(18)
=SS Y) o (o(BLa)) SX Z
fuse the quantization of the next layer into the BN operation of the cur- rent layer, which will not introduce overhead because the BN layer itself requires floating point operations. Note that the float point operations are also element-wise.
Through Eq. 18
16

---

Published as a conference paper at ICLR 2023
Table 6: The comparison between fixed-point operations and float-poin operations for some tasks using the Nearest Neighbor Strategy.
Task
GIN-RE-IB
GCN-MNIST
GAT-CIFARIO
GCN-ZINC
Fixed-poin(MM)
936.96
455.69
1387.98
504.62
Float-point(M)
7.35
2.06
13.71
1.74
Ratio
0.78%
0.45%
0.98%
0.34%
A.4 THE OVERHEAD ANALYSIS OF NEAREST NEIGHBOR STRATEGY
Through our dedicated hardware and the optimized pipeline, we reduce the overhead introduced by the Nearest Neighbor Strategy (NNS) as much as possible. As the pipeline shown in Figure [ fuse the (NNS) with the following operations. The fixed-point results produced by the previous stage are used to first multiply the corresponding step-size from the previous stage (an element- wise float point multiplication) and then execute the NNS process. After getting the step-size, these features are quantized immediately (an element-wis. wise float point multiplication). Therefore, through this fusion way, do not need the extra memory store copy of FP32 features.
In addition, the overhead of the NNS from one more clement-wise float point multiplication and the search process. We provide comparison of the number of float-point operations and fixed-point operations for different graph-level tasks Table where Fixed-point' denotes the fixed-point op- eration, 'Float-point' denotes the float-point operation and the 'Ratio" denotes the percentage of the float-point operations in the overall process. The extra float-point operations introduced by NNS is only tiny fraction of the fixed-point operations. On the other hand, through our optimized pipeline and the comparator array used in our accelerator the latency introduced by the search process of the NNS can be overlapped. Therefore, the overhead introduced by NNS is negligible.
A.5 DATASETS
We show the statistics for each dataset used in work in Table For datasets in node-level tasks, nodes correspond to documents and edges to citations between them. Node features are a bag-of- words representation of the document. The target is classify each node in the graph correctly. The Cora, CiteSeer and PubMed are from Yang et al. (2016). The ogbn-arxiv, ogbl-mag and ogbn-collab are from Hu et al. (2020). The Flickr is from Zeng et al. (2019). The Reddit is from Hamilton et al. (2017). In graph-level tasks, REDDIT-BINARY (Yanardag & Vishwanathan, 2015) is a balanced dataset where each graph corresponds an online discussion thread and the nodes correspond to users. There would be edge between two nodes if at least of them responded to another's comment. The task is then to identify whether a given graph belongs to a question/answer-baseee community
discussion- based community The MNIST and CIFAR-10 datasets (Dwivedi et al. (2020) which are often used for image classification tasks are transformed into graphs in which every node is represented by their superpixel and location, and the edges constructed by Achanta et al. (2012). The task is to classify the image using its graph representation. The ZINC Gömez-Bombare Bombarelli al. (0018) dataset contains graphs representing molecules, where each node is an atom. The task is to regress the penalized logP (also called constrained solubility in some works) given graph. In Figure 8 we show the in-degree distribution for all the datasets we use in our paper.
A.6 EXPERIMENTAL SETUP
To make fair comparison. adopt the same GNN architectures Tailor al. (2020) every task, and the FP32 baseline is also the same. For those tasks that Tailor et al. (2020) does not do, adopt the same architecture their FP32 version. For ogbn-arxiv and PubMed, we use the
17

---

Published as a conference paper at ICLR 2023
Table 7: The statistics for each dataset used in this work.
Task
Name
Graphs
Nodes
Edges
Features
Classes
Cora
1
2708
10556
1433
7
CiteSeer
3327
9104
3703
6
PubMed
1
19717
88648
500
3
ogbn-arxiv
169343
1166243
128
23
Node-Ievel
ogbn-mag
1939743
25582108
128
349
ogbl-collab
1
235868
1285465
128
Reddit
1
232965
1 11606919
602
41
Flickr
89250
899756
500
7
REDDIT-BINARY
2000
429.6
995.5
0
2
MNIST
70000
555
3
10
Graph-Ievel
CIFARIO
60000
117.6
9.41.
5
10
ZINC
12000
23
449.
28
architectures and FP32 results reported by Hu al. (2020) and Kipt & Welling (2016) respectively. We use standard splits for MNIST, CIFAR-TO, T0, and ZINC (Dwivedi et al.) 2000)). For Cora, CiteSeer and PubMed, we use the splits used by Yang et al. (2016). For REDDIT-BINARY, we use 10-fold cross-validation. Our data split way is also the same DQ-INT4.
Figure 9 shows the architectures of the models used in our evaluations, including the layers, number of hidden units, and whether to use skip connection.
Our method implemented using PyTorch Geometric (Fey & Lenssen 2019). We quantize the same parts the DQ-INT4 in all models except for the normalized adjacency matrix in the GCN model, which we have proven that the quantization of this matrix is not necessary in Appendix A..nd proof 2.. The values in the Cora and CiteSeer are all 0 or 1, therefore, we do not quantize the input features for the first layer the GNNs trained on the two datasets as DQ. For all quantized GNNs, consistent
train them by Adam optimizer. The learning rate and the learning rate schedule with their FP32 version. In our method, the quantization parameters (s, b) are also learnable, so we set the learning rate for them, including the for features, for features, and for weights.
When initializing, the parameters of the models are initialized as their FP32 version, the quantization bits for all nodes and weight matrixes are initialized by 4bits, and the step sizes for node features and weights are initialized by s E (0.01, 0.01) except for the graph-level tasks on GAT, where we initialize the step size by s 4(00 1). The distribution. And for GAT model trained on graph-level datasets, we just learn the quantization bits of the node features, while in the attention coefficients computation part, we use the exact 4bit to quantize. The batch size is 128 in all graph-level tasks. The results reported in our work for GNNs on Cora, CiteSeer and Pub Med are averaged over 100 runs with different seeds, and the results for ogbn-arxiv, MNIST, CIFAR-10 10 and ZINC are averaged over ten runs. The results REDDIT. BINARY are obtained by 10-fold cross-validation and the split seed 12345, which is the same RTX 2080Ti GPU driven by Ubuntu 18.04. The
normal distribution and the 24 is uniform
DQ-INT4, All experiments in paper version of the CUDA and Pytorch are 10.2 and 1.8.0, respectively.
A. EXPERIMENTAL SETUPS FOR THE ABLATION STUDY.
The ad advantage of learning- baaned mixed-precision quantization: During the experiment of com- paring the learning bitwidth and bit assignment, ensure the average bits of node features of these two methods are comparable to verify the effectiveness of our A A²Q method. As an example, the average bit is 2.2bit when assigning the bit to nodes with different in-degrees, will first sort the
18

---

Published as a conference paper at ICLR 2023
10
Node Nom
70
Node Num
20x
30'
100
150
60
80
100
In-degree
In-degres
(a) Cora
(b) CiteSeer
10
10
10'
Node Num
Noda Num
10'
10*
1n
10'
10'
20
40
60
100
1.20
140
160
10'
10"
10
10"
Inodlgrve
Inddggro
(c) PubMed
(d) ogbn-arxiv
10
10
Node Num
10
Mode Num
10
10
10
10
3
10
10
15
La dedzege
In degree
(c) MNIST
(f) CIFARIO
10a
107
Node Num
20
Node
10'
10'
16*
1000
2000
3000
(g)
REDDIT-BINARY
(h) ZINC
Figure 8: The in-degree distribution for each dataset used in this work.
nodes by their in-degrees and then select the nodes with the top 20% in-degrees, and quantize those by 3bit, and for the remaining nodes, 2bit to quantize. In the model trained by the bit assign- ment method, the bit not learnable, and other hyperparameters are all consistent with the model using the A 2Q method. For the (GCCN-Coa-miixeddeeeiiiii and CiteSeer-mixed
19

---

Published as a conference paper at ICLR 2023
Dataset
Archi
Cora
CiteSeer
Mub
oghn- arxiv
MNIST
CIFARIO
ZINC
REDDIT-
BINARY
Layers
2
S+IMLP
5-IMLP
S-IMLP
Hidden
GCN
units
16/32/64
16/32/64
16/32/64
256
146
146
145
Skip
NO
NO
NO
NO
YES
YES
YES
connection
Layens
2
S+IMLP
5-IMLP
SHIMPP
5
Hidden
GIN
units
16/32/64
Skip
connection
Layors
SHIMLP
5-IMLP
S+IMLP
Hisdlen
8"brad nu
"head nu
bebud mu mober
19"brad
19"bead
18 1hadd
GAT
umbers
Skip
NO
NO
NO
YES
YES
YES
Figure 9: The model architectures used in our evaluations, the head number for all GAT models on different tasks are 8.
Table 8: The results comparison on GCN-PubMed and GIN-ogbn-arxiv.
Accuracy
Average bits
Compression Ratio
Speedup
GCN(FP32)
78.9#0.7%
1x
PubMed
GCN(DQ)
62.5+2.4%
4
8x
1x
GCN(ours)
77.5=0.1 %
1.90
16.8x
1.45x
GIN(FP32)
68.840.2%
32
1x
ogbn-arxiv
GIN(Q))
57.642.2%
4
8x
GIN(ours)
65.2+0.4%
3.82
8.4x
1.02x
tasks, 3bit and 5bit to quantize the GNNs while keeping the average bitwidth 4bits. In particular, we assign 5bits to those nodes with 50% top in-degrees and assign 3bits to others.
The power of learning the quantization parameters: For the "no-Ir-bit", initialize the bitwidth as 4bits for all nodes features and just train the step size. For the "no-Ir-step". we initialize the step size as previously mentioned but do train them. For the "no-Ir", we just initialize the bitwidth and the step size, but do not train them.
Local Gradient v.s. Global Gradient: All settings of the model trained by global gradient is consistent with the model trained by local gradient method.
The model, without using the Nearest Neighbor Strategy, selects the quantization parameters according to their in-degrees. Every in-degree has corresponding group of quantization parameters. Those nodes whose in-degrees are larger than 1000 will share the same group quantization parameters. In this way, The quantization parameters used by the nodes features can determined as soon as graph data available, without the need for selection during the inference process, and then compare the overhead introduced by the selection process.
The overhead of Nearest Neighbor Strategy:
A.7 MORE EXPERIMENTS RESULTS
This section is a complementary part about experiments results to demonstrate that our A?Q quan- tization method is general and robust.
20

---

Published as a conference paper at ICLR 2023
Table The results comparison on inductive learning tasks and more graphs.
Task
Acc(%)
Average bits
Compression Ratio
30.8+0.1(FP32)
32
GCN-mag
32.r=0.4(Ours)
2.7
11.7x
44.8+1.1(FP32)
32
1x
GCN-collab
(4.ur1.5(OUus))
2.5
12.7x
GraphSage-
95.220.1(FP32)
32
1x
REDDIT
(5ur=
3.9
8.1x
GraphSage-
50.9#1.0(FP32)
32
1x
Flickr
(Ours)
3.8
8.4x
Table 10: Comparison with quantization method.
Task
Acc(%) %
Average Bits
Compression Ratio
16
GCN-Cora
80.9 $0.9=0.6(Ours) 6(0urs)
1.7
9.40x
68.0HO.I(LPGNAA. I(LPGNAS)
8
1x
GAT-CiteSeer
71.9=0.7(Ouns)
1.9
4.21x
12
1x
GraphSage-Cora
74.5+).2(OUrs)
2.7
4.44x
GraphSage-
(9.7HO.3(LPGNAS)
8
1x
Flickr
50.0+0.5(Ours)
3.8
2.11x
50
3000
3000
3000
2500
Acsnge S-dagnos
2000
Nunbos of 5o3ds
Avige 3x dr-ggge
2156
2000
Avammae 2a Jegaes
2000
Nunboa of SuAaaa
80
1824
1500
1.500
1500
1057
1000
1000
1000
20
500
400
500
500
2
B1x
(a) GCN-Cora
(b) GIN-Cora
(c) GAT-Cora
20083
200000
6000
2000
1996)
Nunber modds
Zegree
1500
Natiba of ofrois
Accrage Indogge
200%
129000
Musber of oote
Ancrae Indleges
125086
190000
10300
10:000
2000
19303
7002
13091
0002
S032
1033
400
155
181
19
(d) GCN-PubMed
(e) GAT-PubMed
(f) GCN-ogbn-arxiv
GIN-ogbn-arxiv
Figure 10: The relationship between bit and average in-degrees of nodes using the corresponding bitwidth to quantize. (a), (b) and (c) Three GNN models trained on Cora. (d) and (c) GCN and GAT trained on PubMed, respectively. (f) and (g) GCN and GIN trained ogbn-arxiv, respectively.
21

---

Published as a conference paper at ICLR 2023
Table 11: The effect of #m on the accuracy of quantized model, using GIN trained on REDDIT- BINARY as an example. The average bitwidth is 4bits.
GIN(FP32): 92.2+ 2.3%
GIN(DQ): 81.3* 4.4%
100
400
800
1000
1500
Accuracy 88.7+3.5%
90.6+3.8%
92.0+2.2%
92.6#1.99 9..5+188%
A.7.1 NOVE-LEVEL TASKS
In Table we show more task results on PubMed and ogbn-arxiv. On the GAT-ogbn-arxin task, our GPU raised the Out Of Memory error, so we do not report the results on the GAT-ogbn-arxiv task. The model quantized by our Q method is also significantly better than DQ-INT4, which shows that our A is general. We do not compare with DQ-INTS because our results are comparable with the FP32 baseline with much larger compression ratio than DQ-INT8.
We also show the relationship between bit and average in-degrees of nodes using the corresponding bitwidth quantize more tasks in Figure [0] We present the results of the final layer of GNNs. The results show that the bitwidth learned by our A method is also aggregation-aw aware, which means that our method is robust.
We also evaluate the inductive model, GraphSage, on some other node-level tasks to demonstrate the generality of our method inductive learning tasks. Due to the sampling operation in the GraphSage model, the subgraph input to the model varies, we apply our nearest neighbor strategy to these tasks, i.e., GraphSage-
and GraphSage-Reddit. In addition, we evaluate our method on more datasets, such as the ogbn-mag and ogbl-collab. ogbn-mag is heterogeneous graph and the ogbl-collab is used for the link prediction tasks.
The results of our experiments are presented in Table 9 where we can see that our approach still works well and even brings some generalization performance improvement while significantly com- pressing the model size. This also demonstrates that our Neighbor Nearest Strategy generalizes well on inductive models for node-level tasks.
We also compare with more quantization methods on GNNs. Zhao et al. (2020) uses the Network Architecture Search (NAS) to search for the best quantization strategy for different components in the GNNs. Brennan et al. (2020) explore the use of half-precision (i.e., FP16) in the forward and backward passes of GNNs. Table I0 presents the comparison results on various tasks with these two methods. 'Half- faaff-pre' denotes the method in Brennan et al. (2020), and 'LPGNAS' denotes the method in Zhao et al. (2020). The results demonstrate that our method achieves better accuracy with smaller quantization bitwidth all tasks.
A.7.2 GRAPH-LEVEL TASKS
We propose the Nearest Neighbor Strategy to quantize the node features in graph-level tasks, in which the number of nodes input to models is various. In our Nearest Neighbor Strategy, #mm groups quantization parameters (s, b) should be initialized, and we explore the effect of the value m on the performance of the quantized model in Table [1 using the GIN trained on REDDIT-BINARY dataset. We can observe that when the value m is smaller than 800, the accuracy increases as the value of m increases. When the value of m is higher than 800, the performances of the models with different m are similar. However, the models with a larger m are more stable.
Moreover, the selection of may be related to the number of nodes input to the model. According to our experiments, we finally select m as 1000 for all graph-level tasks.
Table 12 lists the comparison results GIN-ZINC and GAT-ZINC. On the regression tasks, method is also significantly better than DQ-1NT4. Notably, we do not learn different bitwidths for the nodes in ZINC datasets due to the similar topology structure between nodes.
22

---

Published as a conference paper at ICLR 2023
Table 12: The results comparison on GIN-ZINC and GAT-ZINC.
Modle
Dataset
Loss.]
Average bits
Compression Ratio
GAT(FP32)
0.455+0.008
GAT(DQ)
0.520=0.021
8x
GAT(ours)
0.495#0.006
4
8x
ZINC
GIN(FP32)
0.334+0.024
32
GIN(DQ)
0.431=0.012
8x
GIN(ours)
0.380=#.022
8x
3000
3000
L000
3000
Avrrnge In-dagee
2000
Niobtt of noles
Avenue doslape
2000
Awerg Indlgae
2300
Iodogeee
80
11c
116
80
(a) 1-st layer
(b) 2-nd layer
(c) 3-rd layer
(d) 4-th layer
Figure 11: The relationship between bit and average in-degrees of nodes using the corresponding
bitwidth to quantize on different layers of GCN trained on CIFAR1O.
Arergge Io- Po-dogeee
Ninhhr of modes
Avcruat Io-lggre
bodes
Armga Indagae
Arcrge In dogrre
2032
2000
(a) 1-st layer
(b) 2-nd layer
(c) 3-rd layer
(d) 4-th layer
Figure 12: The relationship between bit and average in-degrees of nodes using the corresponding
bitwidth quantize on different layers of GIN trained on CIFAR10.
We also show the relationship between bit and average in-degree of nodes using the correspond- ing bit to quantize for more graph-level tasks different layers immediately after the aggregation phase in Figure Fi-uue 16 The quantization bitwidths learned for graph-level tasks are also aggregation-awwer aware. Because the difference of the in-degrees between different nodes little in the MNIST and CIFAR10 dataset resulting in the aggregated features are similar between different nodes, the relationship between learned bitwidths and the in-degrees is irregular in some layers, e.g., the 2-nd layer in GCN trained MNIST.
23

---

Published as a conference paper at ICLR 2023
2006
10000
1900
1200
Avcagel--dee I-2re
Aecrase b-degr
Aernge belegw
1250
1250
NitPae of
200
(a) 1-st layer
(b) 2-nd layer
(c) 3-rd layer
(d) 4-th layer
Figure 13: The relationship between bit and average in-degrees of nodes using the corresponding
bitwidth to quantize on different layers of GAT trained on CIFARIO.
Antru I-dggns
Aprage Indigne
Aucrue la dagee
Aocue l- degrr
(a) 1-st layer
(b) 2-nd layer
(c) 3-rd layer
(d) 4-th layer
Figure 14: The relationship between bit and average in-degrees of nodes using the corresponding
bitwidth to quantize on different layers of GCN trained on MNIST.
Negg Indegaee
1500
Aecras drgae
Aucaup
(a) 1-st layer
(b) 2-nd layer
(c) 3-rd layer
(d) 4-th layer
Figure 15: The relationship between bit and average in-degrees of nodes using the corresponding
bitwidth to quantize on different layers of GIN trained on MNIST.
1000
Apcmpp le-daees
Nathe coodde
Medra: to Sajece
Aeagg tedagna
(a) 1-st layer
(b) 2-nd layer
(c) 3-rd layer
(d) 4-th layer
Figure 16: The relationship between bit and average in-degrees of nodes using the corresponding
bitwidth to quantize on different layers of GAT trained on MNIST.
24

---

Published as a conference paper at ICLR 2023
Table 13: The impact of the depth of GNNs on quantization performance.
Layers
3
5
Avarage Bits
Avarage Bits
Avarage
Task
Accu(%)
Accu(%)
Accu(%)
Bits
FP32
80.5+0.6
32
79.340.1
32
75.8+3.2
32
GCN-Cora
Ours
80.2+0.6
2.94
78.240.9
3.54
75.0+1.2
3.61
FP32
49.4+15.8
37.1+13.1
32
GIN-Cora
Ours
54.5+12.6
3.3
36.4#11.1
3.1
Table 14: The comparison between the model with and without skip connection on GCN-Cora task.
Without
With
Layers
GCN-Cora
skip connection
skip connection
FP32
Ours
FP32
Ours
Accu(%)
80.5#0.6
80.2+0.6
82.5+0.5
82.2+0.7
3
Bits
32
2.94
32
2.37
Accu(%)
79.3+0.1
78.2+0.9
81.9+0.7
81.5+0.3
4
Bits
32
3.54
32
2.63
Accu(%)
75,8#3.2
75.0+1.2
81.1#1.1
80.620.6
5
Bits
32
3.61
32
2.72
Accu(%) 73.8
73.1
80.1
80,4=0.7
6
Bits
32
4.62
32
2.98
Try
0.5
0.4
2.57
Bitwidth
0.3
0.2
0.1
0.0
Ani moan
Layer
Layer
Figure 17: The average bitwidth for 2nd-5th layer in five layers GCN.
Figure 18: The average bitwidth and quan- tization error for 2nd-6th layer in six layers GCN.
A.7.3 MORE ABLATION STUDY
The impact of the depth of GNNs on quantization performance: We explore how different number of GNN layers impacts the quantization performance of GCN-Cora and GIN-CiteSeer. We explore the guantization performance on 3,4,5.6 layers GCN model and 3,4 layers GIN model (the GCN and GIN used in Table are 2 layers). We did not explore the deeper GNN models because
25

---

Published as a conference paper at ICLR 2023
Table 15: The comparison results on other aggregation funcctinns.
Baseline(FP322
Ours
Bit
Compression Ratio
GIN sum
77.6+1.1%
77.8+1.6%
2.37
13.5x
GIN mean
78.8+011%
78. 78.5+0.6%
2.37
13.5x
GIN max
78.6+..6%
78. 98.6+0.5%
1.97
16.2x
12.5
Sum
Averuge Feature (fferr Aggration)
Mean
10.0
Max
7.5
5.0
2.5
0.0
-25
(150)
[11298
[2120] In-degree
[31,00]
(41.198)
Figure 19; The average aggregated nodes features in different in-degree groups for models with different aggregation functions.
the accuracy of the model decreases drastically as the number of model layers increases due to the over-smooth phenomenon in GNNs. As shown in Table [3] our method can also maintain the performance with a high compression ratio for the model with different layers compared with the FP32 model.
In addition, we observe that the learned quantization bitwidth increases with the number of layers. We analysis the average bitwidth used by 2nd to 5th layer for the five layers GCN model in Figure Our method learns higher bitwidth for the deeper layer. Due to the over-smooth phenomenon that exists the deep layer, the embedding features different nodes are similar the deep layer. Therefore, consider the deeper layer may need a higher quantization bitwidth to distinguish the embedding features of different nodes.
17
The impact of skip connection quantization performance: The first column denoted by "With- out skip connection' and the second column denoted by "With skip connection' [8] present the comparison results for different layers GCN Cora datasets without skip connection and with skip connection, respectively. For the model with skip connection, our method is also effective. Our method learns higher bitwidth for the deeper layer. Due the over-smooth phenomenon that ex- ists in the deep layer, we consider that the deeper layer may need a higher quantization bitwidth to distinguish the embedding features of different nodes. and the higher learned quantization bitwidth for deeper layers also alleviate quantization eтror. And compared to the quantized model with skip connection, the learned quantization bitwidths are higher for the quantized model without skip connection. Figure 18 presents that the quantization errors of the model with skip connection are always higher than the model without skip connection in every layer which means that the model without skip connection is more sensitive to the quantization error. Therefore, a higher quantization bitwidth is necessary for the model without skip connection to maintain the performance. will add these analyses to the appendix in the revision.
Scalability for models that use other aggregation functions: To demonstrate that our method is also helpful the GNNs using other aggregation functions rather than the sum function, replace the aggregation function of the GIN model, which is based on the MPNN framework with mean and max functions, and conduct the comparison experiment the Cora dataset. As shown in Table
26

---

Published as a conference paper at ICLR 2023
Table 16: The comparison reults with the binary quantization method on Cora and CiteSeer datasets.
Accuracy
Average bits
Compression ratio
GCN(FP32)
81.5+0.7%
32
1x
Bi-GCN
81.2+0.8%
1
32x
GCN(ours)
81.4+0.7%
1.61
19.9x
GIN(FP32)
77.6+1.1 %
32
Ix
Cora
Bi-GIN
33.7+6.6%
1
32x
GIN(ours)
77.4=0.8%
1.92
16.7x
GAT(FP32)
83.1=0.4%
32
Ix
Bi-GAT
31.9#0%
1
32x
GAT(ours)
82.6+0.5 %
2.03
15.8x
GCN(FP32)
71.1+0.7%
32
Ix
Bi-GCN
70.7+2.4%
1
32x
GCN(ours)
70.7+0.7%
1.98
16.2x
GIN(FP32)
10.9%
32
Ix
CiteSeer
Bi-GIN
29.1+1.7%
1
32x
GIN(ours)
65.6+1.5%
2.39
13.4x
GAT(FP32) 72.5+0.7%
32
lx
Bi-GAT
20.6+2.6%
1
32x
GAT(ours)
71.0=0.7%
2.15
14.9x
19 the accuracy degradation negligible and the compression ratio is high. indicating that our quantization scheme also applies to the GNNs with mean max aggregation function. We analyze the average features for different aggregation functions in different in-degrees group in Figure 19 The average features of the sum and max functions are highly dependent on in-degrees. The other insight is that the variance of the features is also highly dependent on in-degress.
The analysis demonstrates the generality of our approach, which can capture differences between nodes introduced by topology information of graphs and compress the model size as much possi- ble while maintaining the performance.
A.7.4 COMPARISON WITH BINARY QUANTIZATION METHOD
In this section, show the advantages of our method over the binary quantization method for GNNs. We select the binary quantization method in Wang et al. (2021b) our baseline. We just ran the experiments on the node-level because the binary quantization method only supports node-level tasks, which is one of the drawbacks of the binary quantization method in GNNs. We quantize the same part as Wang al. (202Tb) does for fair comparison.
The comparison results are shown in Table 16 The binary quantization method performs well on GCN, where the aggregation and update phases are simple. However, on both models, GAT and GIN, the accuracy drops significantly compared with the FP32 baseline, which makes the deploy- ment unrealistic. However, our method is immune to this problem, although it has to use higher average bit for node features which believe is necessary for GAT and GIN. In summary, method outperforms the binary quantization method two ways:
1. Our method quantize complex GNN models and ensure the accuracy degradation is negligible compared with the FP32 baseline while achieving a high compression ratio 13.4x- 19.9x.
Our method can be applied to graph-level tasks. However, the binary quantization method can not handle them.
27

---

Published as a conference paper at ICLR 2023
Dats
Cocast sipel
Bde Baller
Boods
Caaad Uai
Waigh Ruffer
DRAM
Fquz Buffer
(a)
(b)
Figure 20: (a) The overview of our accelerator architecture. (b) An example of the bit-serial calcu- lation and the architecture of the MAC.
A.7.5 ACCELERATOR ARCHITECTURE
In this section, we introduce the architecture of our hardware accelerator designed for GNN infer- ence. As presented in Section 3.1 quantize each node feature to appropriate precision and fix the weights to 4bits. To support mixed-precision computation, we adopt bit-serial multipliers at the core. Specifically, we follow the methodology in Judd et al. (2016) only serialize the node features. This way, it takes m cycles to complete the multiplication between an m-bit node feature with 4bit weight, as shown in Figure 20(b) The product involving 2 is implemented by left-shift, i.e. for 2n we can shift left by n bits to implement the product. To increase the computational throughput, we use 256 x 16 MACs which can process 256 16-dimensional features parallel. As shown in Figure 20(a) the compute unit is composed of 256 Processing Engines (PEs), each containing a row of 16 MACs. The architecture of the MAC is shown in Figure 20(b)
The on-chip memory consists of an Edge Buffer, which stores the adjacency matrix of graphs, Weight Buffer, which stores the weight the GNNs, an Input Buffer, and an Output Buffer to store the input features and the output result, and the register of each MAC to store the partial sum. To reduce data movement in the memory hierarchy, the input buffer and output buffer work a swapped fashion, as the output of the current layer is the input the next layer. We set the memory size of Input Buffer, Output Buffer, Edge Buffer, and the Weight Buffer to 2MB, 2MB, 256KB, and 256KB, respectively. The overview of our architecture shown in Figure 20(a)
To calculate B' X'W!, 256 consecutive rows in X and a column of W are mapped onto the MAC array compute 256 inner products in each phase. To achieve this, column of W' is broadcast and shared among PEs. The results of the inner products are written to the output buffer, which can be reused to reduce the off-chip DRAM access. The calculation of X also in inner-produc manner. In this scenario, A is a sparse matrix. We therefore represent A in the Compressed Sparse Row (CSR) format, where full zero rows or elements of A are eliminated. During inference, consecutive compressed rows of A and column of B' are mapped onto the MAC array in each phase. We also sort the nodes in descending order according their in-degrees, and the nodes with similar in-degrees are processed in parallel simultaneously to alleviate the load imbalance problem when performing the aggregation operations.
AB' is
A.7.6 ENERGY EFFICIENCY ANALYSIS
Our method can save energy cost significantly from the following two aspects:
1. By compressing the model size much as possible, e.g., 18.6x compression ratio on GCN-Cora as shown in Table I our method can significantly reduce the memory footprints. Figure 21 presents the energy table for the 45mm technology node. It shows that memory access consumes further more energy than arithmetic operations. Therefore, the memory footprints domains the energy cost, and then compressing the model can much energy cost.
28

---

Published as a conference paper at ICLR 2023
40
Operation
Energy[p]]
Relative Cost
35
bit int ADD
0.03
bit int
0.2
32 bit fooat ADD
0.9
15
32 bit float MULT
3.7
123
32 bit 32KB SRAM
5
167
32 bit DRAM
640
21333
Taalk
Figure 21: The energy table for 45nm tech-
Figure 22: The energy efficiency compared with 2080Ti GPU on various tasks.
nology Hon et al. 2016 Sze 2020
al.
2. Through our quantization method and the accelerator, the model can perform inference using the fixed- peed-pointt operations instead of float-point operations, which are much more energy- mergy-consummng than fixed-point operations. As shown in Figure 21] the 32bit float MULT consumes 18.5x energy compared to the 8bit int MULT. Therefore, our method's energy consumption much lower than the FP32 model.
To illustrate the advantage of our approach in terms of energy efficiency, we compare our accelcrator with the 2080Ti GPU on various tasks. To estimate the energy efficiency of GPU, smi to obtain the power GPU when performing the inference and measure the inference time by time function provided by Python. Then energy cost of our method on the accelerator. We use High Bandwidth Memory (HBM) as our off- chip storage. Then count the number of integer operations, and floating point operations, and the number of accesses to SRAM and HBM when performing the inference process of the quantized models on our accelerator. Based on the data Table 21] we estimate the energy consumed by fixed-point operations and floating-pons operations. The static and dynamic power of SRAM is estimated using CACTI 7.0/aaaaaaalaaaaaraaaao
the nvidia-
get the energy cost of GPU. We also model the
al. (017). The energy of HBM 1.0 is estimated with 7 pJ/bit as in (O'Connor) (2014). Figure 22 presents these results, which shows that the the energy efficiency of our method significantly better than GPU.
29


Reviews:
[{"review_id": "lXMHHeXgW44", "reviewer": "Reviewer_MZPa", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes Aggregation-Aware mixed-precision Quantization (A2Q) method to enable an adaptive learning of quantization parameters, which are innovatively linked to the topology of the graph, thus making more use of the graph information. A Local Gradient method is proposed to train the quantization parameters in semi- supervised learning tasks. For unseen nodes,  Nearest Neighbor Strategy to select quantization parameters. ", "summary_of_the_review": "An article which is nicely motivated and target an unique angle of graph learning: how to more pointedly bake in the topology information to help the effectiveness and efficiency. However, a decent amount of clarification is also needed to further recommend this paper.", "strength_and_weaknesses": "Strength:\n\n1. The idea of (in a more deterministic way) utilizing more of the topology information inside of the graph learning is novel, in my opinion. \n\n2. The experiments conducted are extensive.\n\n3. Nice to see the evaluation using tailored hardware!\n\nWeakness:\n\n1. More clarification about the improvement on compression ratio and speed up. In the article, the compression ratio seems to represent the average bits used by the features as compared to FP32. It is better to show (1) more explicit definition of the metric and (2) the reduction in overall memory consumption of the whole inference/training process. For the speedup, it would be great for the authors to more clearly state which part is assumed to run on the accelerator. For instance, would the nearest neighbor part be executed on the accelerator? \n\n2. For NEAREST NEIGHBOR STRATEGY, it seems the FP32 copy of the features is necessary for \"the feature with the largest absolute value fi in the node features is first selected, and then we find the nearest qmax\"\n\n3. The node-level task accuracy seems too low in Table 1? For instance, https://paperswithcode.com/sota/node-classification-on-cora \n\n4. The authors could provide more illustration on how their methods leverage the graph topology information, as it is a major part of the motivation.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "fH3c5i43Sw", "reviewer": "Reviewer_ZCCp", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an aggregation-aware mixed-precision quantization method for GNNs, which quantizes different nodes features with different learnable quantization parameters, including bit-width and step-size. The paper also proposes a Nearest Neighbor Strategy to deal with the generalization on unseen graphs. Empirical evaluations on several node-level and graph-level datasets demonstrate the efficacy of the proposal.", "summary_of_the_review": "See weaknesses above. I am giving a rating of 3 for now, but I can reconsider my rating based on the authors' rebuttal and discussions with the other reviewers.\n\n*******************************\n\nReview Update: Score changed to 8 after rebuttal.", "strength_and_weaknesses": "Strengths:\n\n1. Given the increasing deployment of GNNs in edge devices, it is important to improve their energy efficiency. This paper tackles this timely problem, and is generally well-motivated.\n\nWeaknesses:\n\n1. My biggest concern is the limited novelty of the paper. It seems that the paper is applying known tricks in mixed-precision quantization to GNNs. The authors add a statement in section 2 \"However,  due to the huge difference between GNNs and CNNs, it is difficult to use these methods on GNNs directly\". However, I am not convinced with such an argument without concrete evidences. What happens if these techniques are naively applied to GNNs?\n2. The paper only reports speedup with their quantization scheme. What about the energy/power consumption which is another important metric given the resource constraints of edge devices?\n3. The authors do not quantify their accuracy-bitwidth trade-off with several related works they cite in section 2. Only DQ-int4 is considered.\n4. Several related comparisons with quantized GNNs are missing, such as [1-3].  \n\n[1] Brennan et al., \"Not Half Bad: Exploring Half-Precision in Graph Convolutional Neural Networks\", 2020\n\n[2] Huang et al., \"VEPQuant: A Graph Neural Network compression approach based on product quantization\", 2022\n\n[3] Zhao et al., \"LEARNED LOW PRECISION GRAPH NEURAL NETWORKS\", 2020", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "emwGxir_wbH", "reviewer": "Reviewer_WF4b", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Motivated by the values of node features that are highly related to the graph structure information such as in-degree, the authors propose and learnable quantization method, $A^{2}Q$, to learn the quantization step size and bitwidth for each node in the graph. Local gradient method is used to overcome the vanishing gradient problem for unlabeled nodes in the semi-supervised node level prediction and a Nearest Neighbor Strategy is used to deal with the generalization on unseen\nfor graph level prediction. Experiments show that $A^{2}Q$ achieves promising results with lower accuracy degradation compared to the previous SOTA on node-level and graph-level tasks.", "summary_of_the_review": "I would like to recommend a score of 7 for this paper. But there are not 7. So I recommend 8 for now. I will adjust my score depending on the rebuttal.", "strength_and_weaknesses": "Strength:\n\n+ The paper is well-written and easy to follow.\n\n+ The proposed Aggregation-Aware Quantization, Local gradient method, and Nearest Neighbor Strategy techniques are well-motivated and novel.\n\n+ Strong experimental results are shown on a variety of datasets on node-level and graph-level tasks.\n\nConcerns and Questions:\n\n- In Figure 1, the average feature of GAT is invariant to in-degree. Why could GAT benefit from Aggregation-Aware Quantization?\n\n- The average feature depends on the chosen aggregation functions. It would be great to do an analysis and experiments with MPNN[1] or GraphSage [2] with different aggregation functions such as mean, max, and sum. I assume only the average feature of sum aggregation will be highly dependent on in-degree. The question is does GNNs with mean or max aggregations also benefit from Aggregation-Aware Quantization?\n\n- The learnable node-wise quantization has significant computation overhead during the training phase. What is the runtime overhead compared to training a regular GNN?\n\n- I am curious about the degradation of quantization for deep GNNs. The depths of GNNs are not mentioned in the experiment section. The quantization error would be accumulated with more layers. I wonder if this method could be used for deeper GNNs with skip connections.\n\n\n[1] Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O. and Dahl, G.E., 2017, July. Neural message passing for quantum chemistry. In International conference on machine learning (pp. 1263-1272). PMLR.\n\n[2] Hamilton, W., Ying, Z. and Leskovec, J., 2017. Inductive representation learning on large graphs. Advances in neural information processing systems, 30.\n\nReference:\n\nA related work on GNN Quantization: VQ-GNN [3].\n\n[3] VQ-GNN: A Universal Framework to Scale up Graph Neural Networks using Vector Quantization\n\n============================================\n\n Post rebuttal:\nThanks for the authors' detailed rebuttal and added experiments. I keep my rating unchanged.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "cXUFUNcYVo", "reviewer": "Reviewer_eih6", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to learn different quantization parameters for each node during GNN training, so as to reduce the quantization bit width and improve the accuracy. By taking the quantization error of each node as a part of the loss function, it solves the problem that many nodes have no gradient. By making statistics of the maximum value of each node feature on an unseen graph and matching the corresponding parameters, the migration between different graphs is realized without retraining.", "summary_of_the_review": "My major concerns are listed in the weakness section 1/2, \"I'm confused how this method could benefit actual scenario\" and \"unclear and important details\". I hope the authors can explain those points.", "strength_and_weaknesses": "==== Strength ====\n* Quantizing the node features according to the topology of the graphs can retain higher algorithm performance is reasonable.\n\n* By taking the quantization error of each node as a part of the loss function, the problem that many nodes have no gradient is solved. This is a new method to solve gradient sparsity in graph neural network transductive learning.\n\n* Good to see the code is provided.\n\n==== Weakness and Questions ====\n\nI'm quite confused about Point 1 below. I'd like to see a thorough reply to it.\n\n1. In which scenarios will the proposed method bring meaningful acceleration?\n\na) Most of the experiments in this paper are transductive learning tasks, which use the information of the whole graph (topology and input features) and labels of some nodes to learn and predict the labels of other nodes. I think for transductive GNN learning tasks, we only need to care about the cost of the training process, since when the training is completed, the original unlabeled nodes \"have been\" predicted, and there is no inference “after the training”. So, I don't quite understand why we need inference-time-quantization-aware training (QAT) here. The only case that may be useful is that after the training, the topology of the graph doesn’t change but the input features of nodes change, which requires re-prediction. But there are no such experiments. And also, it seems the paper doesn’t verify that there is no need to re-learn the quantization parameters in this case (that is, the quantization parameters only depend on the topology of the graph and are independent of the input features of the nodes of the graph).\n\nb) For the inductive learning scenario, where the Nearest Neighbor Strategy proposed in this paper will be used, the model trained on one graph is used to predict the labels of nodes in other graphs. Because the topology information of other graphs cannot be obtained during training, the basic quantization algorithm proposed in this paper can no longer be used. And the proposed NNS strategy needs to perform floating point inference and determine quantization parameters for each node, to minimize the quantization error. I wonder if the float value of each node has been obtained in this process, the node label of this graph can already be predicted. Why do we need a quantization algorithm? \n\n2. The technical details are not clear enough.\n\na) The node-wise quantization scheme sets different quantization steps for each node. Then how to add the features of two nodes with different quantization steps during the aggregation? Do we need to operate with floating-point representation for all additions?\n\nb) It's good to see Figure 2 shows how to perform matrix multiplication of features and weights in the node-wise quantization scheme. But the output y will be the input feature x of the next layer. So how can we transform y with different quantization parameters for each element into a format with different quantization parameters for each node? Also, this transformation is carried out online, and its cost should be counted as the cost of inference.\n\nc) Other operations in GNN are not covered, such as the addition of self-feature and aggregation feature, and batch normalization or node degree normalization after aggregation. How are these operations quantized? For example, the normalization operation will have a greater impact on the data distribution after aggregation.\n\n3. Learnable quantization parameters (bit-width, quantization step) are not new for CNN quantization, should refer to more papers on this.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "8DtP1jJC42", "reviewer": "Reviewer_wtuK", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes Quantization-Aware Training (QAT) method, which utilizes the GNN characteristics, e.g.,  topology and nodes attributes. \nThe proposed method, called  Aggregation-Aware mixed-precision Quantization (A2Q), is automatically learned and assigned\nto each node in the graph different bitwidth (number of bits), allowing heterogeneous allocation of different nodes embedding. Furthermore, to mitigate the vanishing gradient problem caused by sparse connections between nodes, the authors propose a local gradient strategy for exploiting the quantization error of the node features as the supervision during the training procedure. In addition, the authors introduce the Nearest Neighbor Strategy for choosing the appropriate quantization settings for unseen graph nodes. \nExtensive experiments on eight public node-level and graph-level datasets demonstrate the generality and robustness of our proposed method significantly outperform previous art GNN quantization schemes in terms of cpmp[ression ratio, speedup, and model accuracy. \n\n", "summary_of_the_review": "This paper proposes Quantization-Aware Training (QAT) method, which utilizes the GNN characteristics, e.g.,  topology and nodes attributes. \nThe paper is well-written, and the method is well-explained.  \nThe evaluation of the proposed method for quantizing graph-level and node-level tasks shows that the proposed method outperforms previous works in terms of compression ratio, computation complexity, and task performance.\nWhile some part of the proposed method is based on the previous QAT method, For example, the learning of the step size (section 3.1) was previously proposed by [1]. However, the entire method is novel, including the non-homogenous bit allocation (different bitwidth for each node)  and prediction of the quantization parameters for unseen nodes. \n\nWhile overall, I support paper acceptance, there are a few issues that I have pointed out in the weakness section, which I would ask the authors to address during the revision stage. \n\n\n[1] https://openreview.net/pdf?id=rkgO66VKDS\n\n*****Post-rebuttal review*****\nI appreciate the authors for providing a detailed and comprehensive rebuttal and fully responding to my concerns and other reviewers (in my opinion). Hence, I do not doubt that the revised manuscript will contribute to our community, and I would be very happy to see it accepted. My score is also increased to 8.", "strength_and_weaknesses": "*Strengths*\n\nThe paper is well-written, and the method is well-explained.  \nThe evaluation of the proposed method for quantizing graph-level and node-level tasks shows that the proposed method outperforms previous work in terms of compression ratio, computation complexity, and task performance. \nThe proposed method can works also in a supervised/semi-supervised setting, which allows using the proposed procedure for large-scale real-world graphs. In addition, the authors propose a dedicated GNN accelerator architecture to evaluate the compression ratio and the speedup of the quantization. \n\n*Weaknesses*\n\nAlthough authors provided evaluations on various graph datasets, all of them are homophilous and transductive. Please elaborate if the proposed method is scaled to non-homophilous or inductive graphs. Supporting experiments would be essential.  \nThe ablation study of how using a different number of GNN layers impacts the quantization performance is not appearing in the current version of the manuscript. I suggest performing this ablation study during the revision stage.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster