Zero-shot Node Classification with Graph Contrastive Embedding Network
Authors: Wei Ju
Yifang Qin
Siyu Yi
Zhengyang Mao
Kangjie Zheng
Luchen Liu
Xiao Luo
Ming Zhang
Venue: Accepted by TMLR ()

Abstract: This paper studies zero-shot node classification, which aims to predict new classes (i.e., unseen classes) of nodes in a graph. This problem is challenging yet promising in a variety of real-world applications such as social analysis and bioinformatics. The key of zero-shot node classification is to enable the knowledge transfer of nodes from training classes to unseen classes. However, existing methods typically ignore the dependencies between nodes and classes, and fail to be organically integrated in a united way. In this paper, we present a novel framework called the Graph Contrastive Embedding Network (GraphCEN) for zero-shot node classification. Specifically, GraphCEN first constructs an affinity graph to model the relations between the classes. Then the node- and class-level contrastive learning (CL) are proposed to jointly learn node embeddings and class assignments in an end-to-end manner. The two-level CL can be optimized to mutually enhance each other. Extensive experiments indicate that our GraphCEN significantly outperforms the state-of-the-art approaches on multiple challenging benchmark datasets.

Published in Transactions on Machine Learning Research (10/2023)
Zero-shot Node Classification with Graph Contrastive Embedding Network
Wei Ju
juweillpku. edu.cn ginyifang @pku. edu. on siyuyi @mail. nankai. edu. zhengyang mas @stu pku. edu.con on kangjie. zheng Gymaail com liuluchen Opku. edu. on ziookuo Bes. ucla edu mahang os @ypku. edu. on
Yifang Qin
Siyu Y1?
Zhengyang Mao
Kangjic Zheng'
Luchen Liu'
Xiao L 3.*
Ming Zhang
School of Computer Science, National Key Laboratory for Multimedia Information Processing, Peking University
School of Statistics and Data Science, Nonkai University Department of Computer Science, University of California, Los Angeles
Reviewed OpenReview: https:
https:://openrevieuu opeerrview. net/ forum? id= 8uGXn,jRLS
Abstract
This paper studies wero-shot node classification, which aims to predict new classes (i.e., unseen classes) of nodes in graph. This problem is challenging yet promising in variety of real-world applications such as social analysis and bioinformaties. The key idea of zero- shot node classification is to enable the knowledge transfer of nodes from training classes to unseen classes. However, existing methods typically ignore the dependencies between nodes and classes, and fail to be organically integrated in a united way. In this paper, we present novel framework called the Graph Contrastive Embodding Network (GraphCEN) for zero-shot node classification. Specifically, GraphCEN first constructs an affinity graph to model the relations between the classos. Then the node- and class-level contrastive learning (CL) are proposed to jointly learn node embeddings and class assignments in end-to-end manner. The two-level CL can be optimized to mutually enhance each other. Extensive experiments indicate that our GraphCEN significantly outperforms the state-of- the-art approaches on multiple challenging benchmark datasets.
1 Introduction
Zero-shot Learning (ZSL) is an important problem in machine learning, which has been extensively studied in computer vision. It aims at recognizing samples from novel classes that have never appeared in the training data, whose potential has attracted a lot research interests a wide range tasks, such as image classification (Wang et al. 2021a: Li et al. 2022). object recognition (Zablocki al. 2019 Han al., 2020) and knowledge graph completion (Geng et al., 2022; Nayak Bach 2022). To effectively classify those newly emerging classes, existing ZSL approaches typically transfer knowledge from the classes that have training samples (i.e., seen classes) to these unseen classes without requiring any explicitly labeled data for these unseen classes. This can be achieved by the guidance of some auxiliary semantic information (c.g.. category attributes or word embeddings), which usually establishes the mathematian relationship between the semantic space and the embedding space.
* *Corresponding authors.

---

Published in Transactions on Machine Learning Research (10/2023)
Recently, due to the umprecedented success of deep learning, deep neural networks have extended their excellent representation learning capability from the field of vision to graph-structured data. With the advancement of graph neural networks ( (GNNs) [Kipf& & Welling 2017 Ju et al. 2023a Nagarajan & Raghu- nathan 2023 Ma et al. 2023: Chen et al.) 2023 Luo et al. 2023b), node classification, as one of the most graph data analysis, has been widely investigated by various types of GNNs. The basic idea of node classification is to predict the unlabeled nodes with only a small number of labeled nodes on the graph, and it is usually assumed that all classes are covered by the classes of labeled nodes. Never- theless, the graph typically evolves in dynamic and open environments. When newly emerging classes come, previously trained GNNs fail to be discriminative to unseen classes, and massive nodes from novel classes need to be collected and labeled. However, it is costly and laborious to annotate enough samples and retrain the model for all the newly emerging classes. It thus naturally raises a meaningful question: can we recognize the nodes from novel classes that have never appeared?
important problems
Toward this end, zero-shot node classification on graphs has become a promising approach to achieve the goal. Despite the encouraging achievements of previous GNN methods via wessage-passingg mechanism (Gilmer et al. 2017), they still suffer from three key limitations: (i) Under-explored task on graphs. Zero-shot node classification is a newly emerging task that remains largely unexplorod, while existing GNN methods for traditional node classification cannot handle this problem. This is because traditional GNNs are typically trained on a set of labeled nodes and fail to generalize to unseen classes. (ii) Inability to explicitly model the dependencies between nodes and classes. A vast majority of existing GNNs typically learn a mapping function from features to labels to implicitly capture the relationship between nodes and classes, i.c. class information is only incorporated by computing cross-entropy loss with the output embedding vectors. In other words, class labels fail to be explicitly used to enhance information propagation through the model, which is often sub-optimal. Moreover, explicitly modeling such dependencies is crucial for zero-shot node classification, since when facing newly emerging classes, we can leverage these dependencies to effectively align the mapping relationship between the node features and the new class labels, thereby enhancing the model's generalization ability. (iii) Fail to jointly learn node and class embeddings in a united way. Existing approaches mainly concentrate on learning elfective node embeddings for class assignments, while failing to reward the learning of node embeddings from the perspective of classes in turn. As such, we are looking for an approach tailored to zero-shot node classification that can model the dependencies between nodes and classes, and meanwhile jointly learn node and class embeddings in a united way.
To address the above issues, this work proposes a novel framework called the Graph Contrastive Embedding Network (GraphCEN) for zero-shot node classification. The key idea of GraphCEN is to exploit the multi- granularity information to provide sufficient evidence on establishing the relationship between the training classes and newly emerging classes to link cach other. To achieve this goal, GraphCEN first constructs an affinity graph of classes to incorporate category semantic knowledge, we then leverage GNNs effectively integrate the node feature and the class semantics to learn joint information matrix. Grounded in this, we introduce two-level contrastive learning (CT.) based on the joint information matrix, i.e., a node-level CL and class-level CL, respectively. On the one hand, node-level CL is conducted in the row space of the information matrix to learn node embeddings for effective class assignments. On the of oer hand, class-level CL is achieved in the column space of the information matrix to capture compact class embeddings encouraging class-level consistency. Thus the representation learning and class assignment can be jointly optimized to collaborate with each other. By incorporating this mulli-granularity information, our experiments on multiple woal-world datasets prove that our GraphCEN can substantially improve the performance against. existing state-of-the- art approaches. To summarize, the main contributions of this work are follows:
We explore a challenging yet promising problem: zero-shot node classification on graphs, which is under-explored in graph machine learning and data mining.
General Aspects:
Novel Methodologics: We present a novel framework to explore node- and class-level contrastive learning based on the joint information matrix. Node-level CL aims to learn effective node embed- dings, while class-level CL captures discriminative class embeddings.
Multifaceted Experiments: Experimental results on three benchmark datasets demonstrate that our proposed GraphCEN perporos the state-of-the-art approaches.
2

---

Published in Transactions on Machine Learning Research (10/2023)
2 Problem Definition & Preliminary
We first formalize the definition of the graph and define the core problem of our paper: zero-shot node classification. Subsequently, we introduce several types of data augmentation.
Definition 1: Graph. Let 9 EV.E) denote graph, where V = (uI,... , represents the set of nodes, in which N is the number of nodes. & C V x V represents the set of edges. We denote the feature matrix E RRdx where E Rd is the feature of the node U2 and d is the dimension of features. A E (0. (0,1) describes the adjacency matrix, where aij- = 1 if (vi, Uj) E otherwise Cij = 0.
Definition 2: Zero-shot Node Classification. Denote by c (cl....,ce) 9ay) the whole class set, which can be divided into the seen class set C, Ccc...,,cc.... and the unseen class set C. (Ic,++...... 9c) satisfying C, UC,, = C, and C, n. = 0. Let (Si...., denote the semantic description matrix of all classes, cach row of which corresponds to the description of a class. Class semantic descriptions (CSDs) have been detailedly studied by (Wang et al. 2021b). For zero-shot node classification, assuming all the labeled nodes are from seen classes C., based on the CSDs, the goal of zero-shot node classification is to classify the unlabeled nodes whose class set is Cu.
Graph Augmentations. For contrastive learning, data augmentation is crucial that produces novel rational data by applying cerlain transformations without altering the semantics. In partieular, in our experiments, introduce two types of graph transformations via augmenting topological information of the graphs respectively, formalized as:
Edge Dropping. randomly deletes certain ratio of edges in the graph to perturb the edge connectivity pattern where probability follows a default ii.d. uniform distribution. The underlying assumption is that the semantic information of the graph has certain robustness to the edge connectivity pattern variances.
2019 to provide a congruent view of Graph Diffusion. It leverages diffusion (Klicpera et al the original graph, which contributes to offering global information. Here we adopt the Personalize PageRank (PPR) kernel to characterize graph diffussin as
A' ai=a(1-(1 (1- a-) 1ID-I/2(A IJD 1/2)² I)D-1/2)
(1)
where is the teleport probability which is set to 0.2 default. A, D, I represent the adjacency matrix, the degree malrix and the identity matrix, respectively. We can replace A with A' as an augmented view, which can capture long-range neighborhood information.
3 Methodology
3.1
Overview
This paper presents a novel framework GraphCEN for zero-shot node classification as shown in Figure At high level, GraphCEN aims to leverage the multi-gramularity information to jointly learn node embeddings and class assignments in united fashion. Specifically, GraphCEN first constructs class affinity graph to incorporate category semantic knowledge and capture the relations between all classes, and then the joint information matrix is obtained by combining the node fcature and the class semantics via GNNs. Further, the node- and class-level cont rastive learning are respectively conducted in the row and column spaces of the joint information matrix to learn node and class embeddings for effective class assignments. The two steps can be jointly trained collaborate with cach other. In the following, we will claborate on cach component of our proposed framework GraphCEN in turn.
3.2 Class Affinity Graph Construction
A key to zero-shot learning (ZSL) is the acquisition of the class semantic descriptions (CSDs), which serve the auxiliary data to represent the semantics of the classes. However, CSDs of the categories are indepen-
3

---

Published in Transactions on Machine Learning Research (10/2023)
Original Graph
Augmented Graph
GNN Encoder
Node-level Contrast
A
Edge Dropping
Graph Diffusion
GCN Weight
W
Labeled
LCB
+
CCL
Unlabeled
Message Passing
Class-level Contrast
CSDs
Class Affinity Graph
Cls1
A
Cls2
Affinity Graph
Cls3
Construction
Coen
Cls4
Information Fusion
Cls5
Figure 1: Illustraiion of the proposed framework GraphCEN
dently presented based on their own text descriptions or word embedding. Tn other words, the relationship among different classes is ignored, which is crucial for transferring class semantic knowledge from seen classes to unseen classes. As such, how to capture the relationship between different classes, is a key factor for en- hancing the knowledge transfer among all classes and generalization ability under the zero-shot setting.
Therefore, we propose construct class affinity graph to capture the relationship among different classes, such that the semantic information of categories can communicale with each other to allow for semantic knowledge transfer, thus benefiting the better classification for unseen classes.
Specifically, based on the CSDs S = (S1..... (jci.... we construct class affinity graph GA = (VA. EA), where VA = {er, ... } = is the node set consisting of all classes and & is the edge set. ΣA is described by the class adjacency matrix A E R each of whose entry has the form
(e if Eccc)) A
(2)
otherwise
The neighborhood set M(G) of class a; is determined by the k-nearest neighbors of s; in the CSDs.
In this way, the class affinity graph G EA),EA) overcomes the weakness of the independence of CSDs, and the relationship among all classes can be well captured to provide abundant prior knowledge, better serving the zero-shot node classification on graphs. Based on the above class affinity graph, we can leverage a well-designed graph-based encoder to effectively exploit and propagate the semantics of different classes, promoting knowledge transfer from seen classes to unseen classes, which is shown in the following section.
3.3 Information Fusion on Graphs
Existing graph neural networks (GNNs) (Kip' & Welling 2017 Veličković et al., 2017 Ju et al. 2022 Luo
et al.) 2023a) typically model the mapping function from nodes classes through message-passing mecha- nism Gilmer et al. 2017), which encode the structural and attributive information into node embeddings. The key idea is to learn the embedding of each node via aggregating its own embeddings and the ones of its neighbors' along edges. The whole process can be formalized as:
V = ReLU(AXW),
(3)
where V Nx(C| denotes the embedding matrix of nodes. À = -- where À A + and D is the degree matrix of Å. W Rdx(C] is the trainable weight matrix.

---

Published in Transactions on Machine Learning Research (10/2023)
However, vast majority of these GNNs implicitly capture the relationship between nodes and classes via the mapping function, which often leads to sub-optimal solutions. Hence, we argue that explicitly incorporating the relations between the classes and modeling the dependencies between nodes and classes is crucial for effective class assignments. To that effect, on the strength of the constructed class affinity graph A above, we integrate the node feature and the class semantics to learn a joint information matrix defined as:
G = -ReLU(AXW)AS,
(4)
In this way; both the semantics of each node and each class can be efficiently propagated along their respective edges, which contributes to learning the joint information matrix of the nodes and the classes. Moreover, G gxxie] trainable and can be updated for sufficiently combining both sides to simultaneously keep the relations between the nodes and between the classes in united way.
3.4 Two-level Contrastive Learning
Grounded in the joint information matrix of the nodes and the classes defined above, we found a key observation that the rows and columns of the joint information matrix could be respectively regarded the embeddings of nodes and classes. By jointly optimizing the node- and class-level embeddings, we might Iearn effective roprosentations for class assignments, which result in better learning performance.
Inspired by the remarkable success of contrastive learning (CT,), which has demonstrated the strong ability to learn discriminative embeddings from the data itself, we marry this technique into our model for better reprosentation learning. The underlying concept of CL is to explicitly compare pairs of sample embeddings to push away embeddings from different samples while pulling logether those from augmentations of the same sample. Given this, we propose two-level contrastive learning (i.e., node-level and class-level) based on our joint information matrix to jointly optimize node and class embeddings in a united way.
3.4.1 Node-level Contrastive Learning
As the rows of the joint information matrix (i.e., node embeddings of the graph) could be treated as the class assignment probabilities since the dimensionality of the rows equals the number of classos. As such, effective embedding beneficial for generating a more confident class assignment.
Technically, we first randomly choose one of the two graph augmentations in Section to construct node nositive/negatiivy pairs. After encoding them in our well-dosigned GNN of Section 3.3 an additional node projection head (i.e., multi-layer perceptron, MLP) is adopted to map the joint information matrices to the space where contrastive loss is applied. Then we obtain the node-level augmented joint embedding matrices H" E Nxd" and H E Nxd'', Formally, H = ((RLUU(A " A)) and HD = (RRLU (AX))AA)))) where (..) is the projection function with paramcter matrix 11 A = D (AgD 1 Aug, is the augmentation operation, and is the corresponding degree matrix of Aug, (A) for x E {a, b}. Therefore, given 2B augmented node embeddings (hf...., h h....., h22 } from the perspective of the rows, where B is the size of the mini-batch, we expect that the two augmented samples from the same node should be pulled closer, while different nodes should be pushed away. In other words, there are 2B 1 pairs for target node 1g, then we select its corresponding augmented node N to construct a positive pair (hf, h2) and leave other 2B - 2 pairs to form negative pairs. In this way, the training objective of node-level contrastive learning for a given node hy is formulated as:
sim(( ////T
C log B Σ/=1
(5)
sim((;; hg3/T - b2//T)
where I denotes the temperature parameter and sim h2) is the cosine similarity [|bb|]-||b]]]
h hz
Also, two augmented nodes are mirrored and can be switched, hence the total node-level contrastive learning loss is computed over every augmented sample defined as:
B
LXCL = 2B
Σ CC + C.).
(6)
5

---

Published in Transactions on Machine Learning Research (10/2023)
As such, minimizing LNCL achieve the goal that the learned node embeddings have clearer boundaries to distinguish between different sample pairs, such that learned discriminative embeddings are beneficial for effective class assignments.
3.4.2 Class-level Contrastive Learning
In addition to the node embeddingg derived from the rows of the joint information malrix, correspondingly, with a dimensionality of [C], the columns of the matrix can be viewed as class embeddings. In other words, the columns can be interpreted as the distribution of the classes over nodes.
Moreover, the key to zero-shot learning is establishing the mathematical relationship between all classes and transferring knowledge from the seen classes to the unseen classes. Inspired by this motivation, conducting CL on class embeddings can make the different category information well capture the inherent characteristics.
Technically, similar to Section 3.4.1 we obtain the class-level augmented joint embedding matrices Z e N and Z E Nx by encoding the augmented graphs in our well-designed GNN followed by class projection head (i.e., MLP) with parameter matrix θ Therefore, Z can be regarded as the probability of node being assigned to class Given 2|C) augmented node embeddings {zf ...., L.c) zfcp=...... zi..., 2 from the perspective of the columns, where cach embedding zi denotes the i-the column of Z", namely, the embedding of class under the first graph augmentation. Similarly, we encourage the positive pairs to be similar in the embedding space while pushing away the negative pairs. Thus we can leverage the idea of CL. to define the training objective of class-level contrastive learning as:
xim(
G -log
(7)
|C| f=1
(c cim(a
By traversing all classes, the total class-level contrastive learning loss computed as:
|el
LCC..
Σ
C + -1) -Z),
(8)
2/C|
where (-) represents the entropy function to prevent collapsing into trivial outputs of the same class. Math- ematically, (Z) D -D lo - p log p where Df = -'- / |2 x E {a, b} the normalized probability vector. Here, compared with learning discriminative class representations through contrastive learning to implicitly prevent model collapse, we directly introduce the entropy function to encourage each column of the augmented joint embedding matrix to be close to uniform distribution, explicitly avoiding the trivial solution that most nodes are assigned to the same class (Hu et al. 2017
In this way, class-level contrastive learning can reward the optimization of node-level contrastive learning, and guide the learning of node embeddings for effective class assignments.
3.5 Training and Optimization
To jointly integrate the two-level contrastive learning, we combine the two contrastive losses to collaborate with each other. And the training objective can be written as:
Lel = ENCL + LCCL-
(9)
Moreover, we can also incorporate supervision signals from the labeled nodes from seen classes C., where the cross-entropy loss function can be defined as:
C. tiue In Y.j.
LCB
(10)
where Y, softmax softmax(AXX) denotes the predicting probability of the i-th node belonging to class j. is the number of the labeled nodes.

---

Published in Transactions on Machine Learning Research (10/2023)
Algorithm 1: Optimization Algorithm of GraphCEN
Input Graph data G (V,E, classes semantic descriptions matrix S, total classe set C, seen classes Cs, trainable weight matrix W GNN, parameter matrix θ= = [O 0 in two projection heads, temperature parameter and contrastive parameter Output: Class assignments for unlabeled nodes from unseen class C.
Construct the class affinity graph GA by Eq. 2
Initializing trainable weight matrix W and parameter matrix O
while not done do Compute the joint information matrix G by Eq. Sample a mini-batch of nodes from 9;
Sample one augmentation from Section 2
Compute node-level contrastive loss LNCL. by Eq.
Compute class-level contrastive loss Locr, by Eq.
Compute the supervised loss Lce by Eq. 10
end Update parameters W and + by gradient descent to minimize & by Eq. 11
Finally, by combining supervised loss Lee with two-level contrastive loss LCL, the overall loss function of our GraphCEN can be calculated as:
L L= + LCL.
(11)
where B is the tuning parameter controlling the magnitude of supervised loss and two-level contrastive loss. We summarize the optimization algorithm for our GraphCEN in Algorithm I
3.6 Computational Complexity Analysis
Suppose that N is the number of nodes, |8| the number of edges, B is the batch size, d is the dimension of original node features, and d' is the dimension of the node-level joint embedding matrices after a projection head. We compute the joint information matrix in O(EE| Nd[[] + NN(C)2) based on sparse adjacency matrix and the complexity of node- and class-level projection heads is O(Ndd' + Nd(C)). Moreover, the complexities calculating node- and class-level contrastive losses for each batch are O(B'd') and O((C)²),, respectively. Therefore, the total computational complexity is O(N N (B2d + B)) = O((E) N(CC)d (B d)d' + 2()
NNdCC)
4 Experiments
4.1 Experimental Setup
Datasets. For comprehensive comparisons, we conduct extensive experiments on three real-world citation datasets, which are Cora (McCallum et al., 2000), Citeseer (Giles et al. 1998), C-M10M (Pan et al., 2016 and a large-scale dataset ogbn-arxiv (Wang et al. 2020).
In these four datasets, nodes represent different publications, and edges represent the citation relationship between the linked two publications. The specific class labels the datascts are displayed in Table I For the zero-shot node classification, we follow the same seen seen/unseen /unseen class split settings introduced in (Wang el al. (021b) to make a fair comparison as shown in Table [ Besides, we adopt CSDs (default) and LABEL- generated by Bert-Tiny as auxiliary data following (Wang et al. 2021b). which can provide the semantic information of different labels. We will further compare the dillerences
kinds of CSDs, i.e., TEXT-
between these CSDs in Section 4.5
Baseline Methods, On the one hand, we compare our GraphCEN with various state-of-the-as zero-shot learning methods including DAP and its variant DAP(CNN) (Lampert et al. 2013), ESZSL (Romera-Parooee & Torr, (005)). ZS-GCN and its variant ZS-GCN(CNN) (Wang et al. 2018). WDVSe (Wan et al. 20199). and Hyperbolic-7ST. (Liu et al. 2020), which are primarily proposed for vision domains. On the other hand.
7

---

Published in Transactions on Machine Learning Research (10/2023)
Table 1: The data split and class information of three citation datasets.
Class Split 1
Class Split II Train//all /
Dataset
Classes
Class Labels
[Train/Vaas /aal//Va/aa/aa Test]
Neural Network, Rule Learning,
Cora
7
[3/0/4]
[2///3]
Reinforcement Learning, Probabilistic Methods,
Thcory, Genetic Algorithms, Cased based
Agent, Information Retrieval,
Citeseer
6
[2/0/4]
[2/2/2]
Database, Human Computer Interaction, Artificial Intelligence, Machine Learning
Biology, Computer Science, Finacial Economics, Industrial Engineering, Physics, Social Science
C-M10M
[3/0/3]
[2/2/2]
ogbn-arxiv 40
[20/0/20]
[13/18/14]
As. cs. cs.OS,
two recent methods designed for zero-shot node classification, DGPN (Wagg et al. 2021b and DBiGCN Yuue et al., 2022), are also considered for comparison. In addition, we adopt the RandomGuess as the naive baseline, which guesses the unseen labels for the unlabeled nodes randomly.
Implementation Details. For our GraphCEN, we adopt the grid search for the parameters un- der the class split I, and determine the optimal hyper-parameters by the validation classes under follows: the learning rate
the class split I1. The space of typer-parameters is carefully selected E {0.0005, 0.001, 0.005, 0.01}, the number of hidden units E f32, 64, 128, 256), the temperature paramcter E (0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0), the contrastive parameter 3 E (0.05, 0.1, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0). Besides, the framework is trained using Adam optimizer (Kingma Ba 2014). In our experiment, we use accuracy as the common evaluation metric to evaluate the performance.
4.2 Experimental Results
In this section, we evaluate the performance of all the algorithms for zero-shot node classification. The results are summarized in Table According the quantilative results, we have the following observations:
Overall, from the results, it can be observed that our framework GraphCEN achieves the best performance against other strong baselines on all three datasets under different class split settings. In particular, GraphCEN outperforms the closest competitor 7.43% on Cora under the class split and 7.12% on C-M10M under the class split. II, which demonstrates the excellent capability of our framework for zero-shot node classification.
Traditional zero-shot learning methods generally perform worse than the recent methods designed for zero-shot node classification. Maybe the reason is that traditional zero-shot learning methods make it difficult capture the characteristics of complex graph-structured data, while DPGN and DBiGON excel al exploring the relational information between nodes in graph datasets.
For all datasets, our proposed GraphCEN outperforms strong baselines DPGN and DBiGCN by a significant margin, which suggests that our approach more effective capturing dependencies between nodes and nodes, nodes and classes. The two-level contrastive learning can also inherently learn the representations with generalization ability from the joint information matrix, which is more beneficial for the knowledge transfer from seen classes to unseen classes.
To validate the scalability of our proposed method, we conduct experiments on the largo-scale dataset. ogbn-arxiv. We compare it with the two latest baselines DPGN and DBiGCN, and from
8

---

Published in Transactions on Machine Learning Research (10/2023)
Table 2: The overall performance (%) on three benchmark datasets for zero-shot node classification, The best results are shown in boldface and the second-best is underlined.
Cora
Citescer
C-M10M
RandomGuess
25.35
24.86
33.21
DAP
26.56
34.01
38.71
DAP(CNN)
27.80
30.45
32.97
ESZSL
27.35
30.32
37.00
Class Split
ZS-GCN
25.73
28.62
37.89
7S-GC)(CNN)
16.01
21.18
36.44
WDVSc
30.62
23.46
38.12
Hyperbolic-7SL.
26.36
34.18
35.80
DGPN
33.76
37.74
41.93
DBiGCN
45.08
38.57
41.11
GraphCEN (Ours)
48.43
40.77
44.17
Improve
+7.43%
+5.70%
+5.34%
RandomGucss
32.69
50.48
49.73
DAP
30.22
53.30
46.79
DAP(CNN)
29.83
50.07
46.29
Class Split II
ESZSL
38.82
55.32
56.07
ZS-GCN
29.53
52.22
55.28
7S-GCN(CNN)
33.20
49.27
51.37
WDVSc
34.13
52.70
46.26
Hyperbolic-ZSL.
37.02
46.27
55.07
DGPN
48.31
58.86
61.68
DBiGCN
46.95
58.37
66.12
GraphCEN (Ours)
50.61
60.47
70.83
Improve +
+4.76%
+2.74%
+7.12%
Table 3: The comparison (%) of DGPN, DBiGCN, and our proposed GraphCEN on the large-scale ogbn- arxiv dataset for zero-shot node classification.
DGPN
DBiGCN
GraphCEN (Ours)
Class Split
22.37
21.40
23.96
Class Split II
21.95
25.92
28.36
Table it can be that method GraphCEN still outperforms others the largo-scale datasel, particularly in the class split II, highlighting the strong discriminative ability and better generalization achieved through our dual-contrastive learning. This fully demonstrates that our method GraphCEN learns more powerful representations and exhibits better scalability.
4.3 Ablation Study
To provide further insights into the GraphCEN, we conduct the ablation study to evaluate the effectiveness of the three main components, i.c., node-level contrast (N-level) and class-level contrast (C-level + Eutropy). Next, we explore six different variants designed as follows:
My: Our base model, which trains GNN solely on labeled data in a fully supervisod manner with training objective LcE;

---

Published in Transactions on Machine Learning Research (10/2023)
Table 4: Analysis of ablation study. N-level, C-level and Entropy represent that node-level contrastive loss, class-level contrastive loss or the entropy function is applied, respectively.
Contrastive Item
Accuracy (%)
N-level
C-level Entropy
Cora
Citeseer
C-M10M
M
45.47
38.18
33.38
M
45.94
38.71
33.52
M3
V
46.14
38.81
33.47
M4
V
47.96
39.25
36.85
Ms
48.09
39.60
43.71
Ms
V
48.43
40.77
44.17
M2: is a variant with only class-level contrast except for the entropy function where we optimize the GNN with both training objectives Lce and LOCL + (Z):
My: It is variant with only class-level contrast where we optimize the GNN with both training objectives Lee and Leet;
M4: It is variant with only node-level contrast where we optimize the GNN with both training objectives Lee and LNCL;
My: It is variant with both node-level and class-level contrasts except for the entropy function where optimize the GNN with both training objectives Lce and LNCL + (Z);
Mg: Our full model, which combines both node-level and class-level contrasts with both training objectives Lce and LCL.
The results of the above six variants are summarized in Table The performance of models M3 and M1 shows the superiority over model M1 all datasets, indicating that the introduction of both class-level contrast and node-level contrast alone indeed improve overall performance. For all datasets, the performance of model Mg which incorporates both class-level and node-level contrasts is superior to the other variants. We can infer that either node-level or class-level contrast is indispensable, which implies that the combination of both levels leads to more discriminative and generalizable node embeddings for effective class assignments, thus being beneficial for zero-shot node classification. Additionally, when comparing M2 with M3 and Ms with Mc, we observe that the increase in Entropy is relatively small, except when contrasting Ms and Mo the Citescer. The reason might be that class-level contrast already ensures the learning of discriminative class representations, implicitly preventing collapse to trivial solutions. However, considering that Entropy can further guarantee the model's optimization for different datasets, playing an explicit protective role, we choose to retain this term, and the experimental gains also demonstrate the necessity of Entropy.
4.4 Sensitivity Analysis
In this section, we look into the scnsitivity of parameters: contrastive weight B, contrastive temperature т.
Effect of Contrastive Weight. We first examine the impact of the contrastive weight on two datasots Cora and Citeseer by varying 3 from 0.0 to 10.0. As depicted Figure a(a), the performance on Cora improves gradually when 33 increases from 0.0 0.1 and remains stable from 0.2 to 2.0. We attribute this improvement to the introduction of both class- and node-level contrastive lcarning. Nevertheless, excessive contrastive weight Cora may harm performance due to the neglect of the supervised loss, biasing the correct direction of gradient learning. Additionally, the performance on Citeseer exhibits similar trend of improvement when 3 increases from 0.0 to 0.2 However, our approach is not sensitive to parameter variations from 2.0 to 5.0 on Citesecr.
Effect of Contrastive Temperature. We then conduct experiments on datasets Cora and Citeseer evaluate the sensitivity to rastive temperature As shown in Figure bb), the accuracy both datasets
10

---

Published in Transactions on Machine Learning Research (10/2023)
Results on Cora
Results on Citeseer
0.500
0.450
0.450
0,400
Aceuracy
Accuracy
0.400
0.350
0.350
0.300
0.250
0.05
0.1
0.2
0.5
1.0
2.0
5.0
10.0
0.05
0.1
0.2
0.5
1.0
2.0
10.0
Contrastive Weight B
Contrastive Weight B
(a) Analysis of contrastive weight B. Results Cora
Results Citeseer
0.500
0,450
0,450
Accuracy
Accuracy
0.400
0.350
0.350
0.300
0.300
0,250
0.05
0.1
0.2
0.5
1.0
2.0
5.0
10.0
0.05
0.1
0.2
0.5
1.0
2.0
5.0
10.0
Contrastive Temperature
Contrastive Temperature T
(b) Analysis of temperature parameter т.
Figure 2: Analysis of parameter 3 and in contrastive learning.
Table 5: Accuracy (%) of zero-shot node classification w.r.t. different CSDs on Cora.
Cora
TEXT-CSDs LABEL-CSD: Decline rate
DAP
26.56
25.34
-4.59%
Class Split I
ESZST.
27.35
25.79
-5.70%
ZS-GCN
25.73
23.73
-7.77%
WDVSc
30.62
18.73
-38.83%
ZSSL
26.36
25.47
-3.38%
DGPN
33.76
32.69
-3.17%
DBiGCN
45.08
32.89
-27.04%
GraphCEN (Ours)
48.43
39.63
-18.07%
reaches its peak when J set to 0.2, and gradually decreases as 7 continues to increase. This is because higher temperature will result in softer distribution, where all classes are more likely, while lower temperature will result in a sharper distribution, where some classes are much more likely than others. In other words, higher temperature would make the model less confident in its predictions, which would result in lower accuracy.
4.5 Discussion on Different CSDs
Since there are two kinds of CSDs, we can construct the class affinity graph defined in Eq. 2 based on them. As the two CSDs provide different semantic information for the class affinity graph GAi the model performance on zcro-shot node classification, which highly dependent on the label semantics, would be significantly affected by the chosen CSDs. Specifically, the accuracy of zero-shot node classification w.r.t. the used CSD type is shown in Table and 7 It can be observed that the performance with TEXT-CSDs is
11

---

Published in Transactions on Machine Learning Research (10/2023)
Table 6: Accuracy (%) of zero-shot node classification w.r.t. different CSDs on Citeseer.
Citeseer
TEXT-CSDs LABEL-CSDs Decline rate
DAP
34.01
30.01
-11.76%
Class Split 1
ESZST.
30.32
28.52
-5.94%
ZS-GCN
28.62
26.11
-8.77%
WDVSc
23.46
19.70
-16.02%
Hyperbolic-ZSL ZSL
34.18
21.04
-38.44%
DGPN
37.74
31.05
-17.73%
DBiGCN
38.57
34.18
-11.38%
GraphCEN (Ours)
40.77
38.45
-5.69%
Table 7: Accuracy (%) of zero-shot node classification w.r.t. different CSDs on C-M1O.
C-M1OM
TEXT-CSDs
LABEL-CSD: Decline rate
DAP
38.71
32.67
-1..00%
Class Split I
ESZSL
37.00
35.02
-5.35%
ZS-GCN
37.89
33.32
-12.06%
WDVSc
38.12
30.82
-19.15%
Hyperbolic-7ST.
35.80
34.49
-3.66%
DGPN
41.93
35.12
-16.24%
DBiGCN
41.11
37.54
-8.68%
GraphCEN (Ours)
44.17
38.68
-12.43%
higher than with LABEL-CSDss which is consistent with previous researches (Wang et al. 2021b Yue al. 2022). The result shows the expressiveness of natural languago when comes representing label-label relationships. Among the listed models, GraphCEN achieves the best performance no malter which kind of CSDs are used, which further demonstrates the robustness of the proposed approach via incorporating the two-level contrastive learning.
4.6 Analysis of Graph Augmentation
To explore the ellect of different graph augmentations, we compare the performance three stralegies, i.e., original graph, edge dropping, and graph diffusion. The horizontal axis denotes the augmentation strategy applied to generate H while the vertical axis represents the strategy for generating H². The results on two datasets are reported in Figure 3 It can be observed that different augmentation strategies exhibit vital effects. GraphCEN with (ED GD} obtains the best performance, whereas {OG + OG) (without augmentations) yields the worst results. Maybe the reason is that different augmentations can maximize the diversity of graph data while preserving semantics, which better promotes consistency in contrastive learning and enhances generalization for zero-shot node classification. Moreover, we find that the effect of (GD + GD) is worse than that of (GD + OG). The possible reason the generation of graph diffusion is not random, resulting in the same augmented views, which hinders the diversity of the augmentation process, while {ED ED} with randomness will lead to different augmented views, and thus performs better than that of graph diffusion. Besides, augmentation with (ED OG} is worse than {GD + OG). The explanation for this is that graph diffusion is likely to capture the global information instcad of exploring from a local view like edge dropping, which results in inadequate semantic information exploration. Overall, the aforementioned experiments demonstrate the importance graph augmentation in contrastive learning. Encouraging consistency with diverse data augmentations maximizes the model's ability learn robust semantic representations, enhances its generalization capabilities, and proves beneficial for our zero-shot node classification task. If only the original graphs are used without augmentation, the model may become highly
12

---

Published in Transactions on Machine Learning Research (10/2023)
Cora
Citeseer
OG
45.34
OG
39.16
430
465
ED
46.41
47.42
ED
40.28
40.09
GD
47.62
48.43
47.09
GD
40.38
40.77
39.99
353
OG
ED
GD
OG
ED
GD
Figure 3: Analysis of different graph augmentations on datasets Cora and Citeseer. OG, ED, and GD correspond to original graph, odge dropping, and graph diffusion, respectively. Here the horizontal axis denotes the augmentation strategy applied generate H while the vertical axis represents the strategy for generating H
sensitive to data perturbations variations, hampering its ability to learn discriminative representations and consequently affecting the model's performance.
5 Related Work
Node Classification. As one of the most fundamental problems in relational data modeling, many of the real-life applications can be boiled down to the problem of semi-supervised node classification. It aims to predict the unlabeled nodes with only few labeled nodes on the graph. This problem has been extensively
investigated with graph neural networks (GNNs) (Ju et al.] 2023a Nagarajan & Raghunathan
2023 Ma
et al. 2023 Prieto et al. 2023 Zhong el al. 2022 Chen et al.
2023 Ju et al.] 2023d; Demirel et al. 2021
and can generalize well to downstream domains such as social analysis (Bhagat et al. 2011 Yuan et al,
2023 and bioinformatics Yuuan al. 2021 Gasteiger et al.,
2022). Benefiting from the powerful capability
incorporate topological structure and associated features, GNNs have shown outstanding performance in learning effective node embeddings. Nevertheless, these methods have the inability to handle zero-shot node classification, while our proposed GraphCEN models the dependencies between nodes and classes to transfer knowledge from the seen classes to the unseen classes.
Zero-shot Learning. Our work essentially belongs to zero-shot learning (ZSL), which has been extensively studied in computer vision. Inspired by human cognitive competence, this topic has recently intrigued vast interest, with the capability of recognizing new classes during learning by exploiting the intrinsic semantic relationship between seen and unseen classes, with the guidance of some auxiliary information. Existing ZSL methods can be divided into two main categories: (i) embedding- based methods (Akata et al. 2015 Frome
et al. 2013 Lampert et al. 2013), and (ii) generative- based methods (Xian et al. 2018 2019 Schonfeld et al, 2019 Guan et al. 2020). For the first category, it aims to learn an embedding mapping function from visual space to semantic space. For the second category, the basic idea is to apply generative model to senthesize features of unseen classes. However, these methods are not tailored to zero-shot node classification, and fail to capable of processing complex data structures, such as graph domains. Recently, DGPN (Wang et al., 2021b) and DBIGCN (Yue et al., (022)) have been proposed for zero-shot node classification. DGPN designs the acquisition of high-quality class semantic descriptions (CSDs) and follows the principles of locality and compositionality, while DBiGCN proposes two dual GCNs with two opposite directions and introduces label consistency loss to mutually enhance. Different from them, our proposed model GraphCEN starts from the relationship between nodes and classes, and utilizes contrastive learning to mine the intrinsic connections between semantics, thus promoting knowledge transfer more effectively.
Contrastive Learning. Another category of related work contrastive learning (CL), which has recently achieved stato-of-the-art performance in the direction of self-supervised learning, arousing extensive attention from researchers. The core idea of CT. is based the task of instance discrimination (Wu et al. 2018) and
13

---

Published in Transactions on Machine Learning Research (10/2023)
introducing a point contrastive loss that encourages the matched positive point pairs to be similar in the embedding space while pushing away the negative pairs (Hadsell et al. 2006). Recently, there are many CT. approaches proposed to capture the discriminative representations (Chen et al. 2020a He et al., 2020 Chen ct al. 2020b; Grill et al. 2020). For example, SimCLR (Chen et al. 2020a) trains an encoder by adopting multiple data augmentations and learnable nonlinear translormation to pull the feature embeddings from the same images. MoCo (He et al. 2020) develops dynamic dictionary for CL via a moving-averagod encoder. SupCon (Khosla al. 2020) extends the self-supervised contrastive approach to the fully supervised setting, which allows for effective leverage of the label information. Additionally, many recent methods are extending CL to graph domains (You et al. 2020 Ju et al. 2023c; Yi et al. 2023: Luo et al. 2022 Ju et al. 2023b). Compared with existing CL methods, our work goes further and explores the node- and class-level
CL simultaneously for zero-shot node classification on graphs.
6 Conclusion
In this paper, develop novel framework GraphCEN for zero-shot node classification, which explicitly models the dependencies between nodes and classes to transfer knowledge from seen classes to unscen classes. We first construct a class affinity graph to capture similar category semantics, and then integrate the class affinity graph and node features into the joint information matrix via GNNs. Further, we introduce node- and class-level contrastive learning based on the information matrix for effective class assigmments. Experimental results demonstrate that our proposed GraphCEN consistently outperforms existing state-of-thh-art methods.
As our future work, there several aspects of the proposed model that deserve further investigation: (i) leveraging the neighbor information of constructed class affinity graph to conduct better contrastive learning; (2) adopting the unsupervised pre-training techniques to sulficiently explore inherent graph semantics for better knowledge transfer; (iii) extending our framework to more challenging settings such as generalized zero-shot learning or open-world graph learning.
Acknowledgments
This paper is partially supported by the National Natural Science Foundation of China with Grant (NSFC Grant No. 62106008, No. 62276002 and 62306014) well the China Postdoctoral Science Foundation with Grant No. 2023M730057.
References
Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. Tabel-embeddiing for image clas- sification. IEEE transactions on pattern analysis and machine intelligence, 38(7):1425 1438, 2015.
Smriti Bhagat, Graham Cormode, and S Muthukrishnan. Node classification in social networks. In Social nelwork dala analylics, pp. 115 148. Springer, 2011.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020a.
Xinlci Chen, Haoqi Fan, Ross Girshick, and Kaiming Hc. Improved basclines with momentum contrastive learning. arXio preprint #xXx:2003.0429,, 2020b.
Yilan Chen, Tianning Xu, Dilek Hakkani-Tur, Di Jin, Yun Yang, and Ruoqing Zhu. Calibrate and debias layer-wise sampling for graph convolutional networks. Transactions on Machine Learning Research, 2023.
Mchmet F Demirel, Shengchao Liu, Siddhant Garg, Zhenmei Shi, and Yingyu Liang. Attentive walk- aggregating graph neural networks. Transactions on Machine Learning Research, 2021.
A Frome, GS Corrado, J Shlens, al. deep visual-semantic embedding model. Proceedings of the Advances in Neural Information Processing Systems, pp. 2121-211 2121-2129, 2013.
14

---

Published in Transactions on Machine Learning Research (10/2023)
Johannes Gasteiger, Muhammed Shuaibi, Amuroop Sriram, Stephan Günnemann, Zachary Ulissi, C Lawrence Zitnick; and Abhishek Das. Gemnet-oc: developing graph neural networks for large and diverse molecular simulation datasets. Transactions on Machine Learning Research, 2022.
Yuxia Geng, Jiaoyan Chen, Wen Zhang, Yajing Xu, Zhuo Chen, Jeff 7. Pan, Yufeng Huang, Feiyu Xiong, and Huajun Chen. Discntangled ontology embedding for zcro-shot learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Dala Mining, pp. 443 -43,, 2022.
C Lee Giles, Kurt D Bollacker, and Steve Lawrence. Citesoer: An automatic citation indexing system. In Proceedings of the third ACM conference on Digital libruries, pp- 89 98, 1998.
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry, In International conference on machine leorning, pp. 1263 1272. PMLR, 2017.
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271-21284, 2020.
Jiechao Guan, Zhiwu Lu, Tao Xiang, Aoxue Li, An Zhao, and Ji-Rong Wen. Zero and few shot learning with semantic feature synthesis and competitive learning. IEEE transactions on pattern analysis and machine intelligence, (3(7):2510-25) 2523, 2020.
Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Compuler Sociely Conference Compuler Vision and Pollern Recognilion (CVPR 06). volume 2, pp. 1735-17425 IEEE, 2006.
Zongyan Han, Zhenyong Fu, and Jian Yang. Learning the redundancy-free features for generalized zero- shot object recognition. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 12865 12874, 2020.
Kaiming Hc, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervisod visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729-9773 9738, 2020.
Weihua Hu, Takeru Miyato, Seiya Tokui, Eichi Matsumoto, and Masashi Sugiyama. Learning discrete rep- resentations via information maximizing self-augmented training. In International conference on machine learning, pp. 1558-1567. PMLR, 2017.
Wei Ju, Junwei Yang, Meng Qu, Weiping Song, Jianhao Shen, and Ming Zhang. Kgnn: Harnessing kernel- based networks for semi-supervised graph classification. In Proceedings of the fifteenth ACM international conference on web search and data mining, pp. 421-429. 2022.
Wei Ju, Zheng Fang, Yiyang Gu, Zequn Liu, Qingqing Long, Ziyue Qiao, Yifang Qin, Jianhao Shen, Fang Sun, Zhiping Xiao, et al. A comprehensive survey on deep graph representation learning, arXiv preprint 3rXiv:2304.0505: 2023a.
Wei Ju, Yiyang Gu, Binqi Chen, Gongbo Sun, Yifang Qin, Xingyuming Liu, Xiao Luo, and Ming Zhang. Glee: A general framework for graph-level clustering. In Proceedings of the AAAI Conference on Arlificial Intelligence, volume 37, pp. 4391-4399, 2023b.
Wei Ju, Zequn Liu, Yifang Qin, Bin Feng, Chen Wang, Zhihui Guo, Xiao Luo, and Ming Zhang, Few-shot molecular property prediction via hierarchically structured learning relation graphs. Neural Networks, 163:122 131, 2023c.
Wei Ju, Xiao Luo, Meng Qu, Yifan Wang, Chong Chen, Minghua Deng, Xian-Sheng Hua, and Ming Zhang. Tgnn: A joint supe-super framework for graph-level classification. arXiv preprint arXiv:2304 11688, 2023d.
15

---

Published in Transactions on Machine Learning Research (10/2023)
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Newral Information Processing Systems, 33:18661 18673,, 2020.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ar preprinl 1rXiv:141..690 2014.
Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In ICLR. 2017.
Johannes Kliepera, Stefan WeiBenberger, and Stephan Günnemann. Diffusion improves graph learning. arXiv preprint arXii::911.05485, 2019.
Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. Attribute-based classification for zero-shot visual object categorization. IEEE transactions pattern analysis and machine intelligence, 36(3):453- 465, 2013.
Xiangyu Li, Xu Yang, Kun Wei, Cheng Deng, and Muli Yang. Siamese contrastive embedding network for compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2022.
Shaoteng Liu, Jingjing Chen, Liangming Pan, Chong- Wah Ngo, Tat-Seng Chua, and Yu-Gang Jiang. Hy- perbolic visual embedding learning for zero-shot recognition. In Proceedings of the IEEE/CVP conference on computer vision and pottern recognition, pp. 9273-9281, 2020.
Xiao Luo, Wei Ju, Meng Qu, Yiyang Gu, Chong Chen, Minghua Deng, Xian-Sheng Hua, and Ming Zhang, Clear: Cluster-cnhaneed contrast for solf-supervieed graph representation learning. IEEE Transactions on Newral Networks and Learning Systems, 2022.
Xiao Luo, Wei Ju, Yiyang Gu, Zhengyang Mao, Luchen Liu, Yuhui Yuan, and Ming Zhang, Self-supervised graph-level representation learning with adversarial contrastive learning, ACM Transactions on Knowledge Discovery from Data, 2023a.
Xiao Luo, Yusheng Zhao, Zhengyang Mao, Yifang Qin, Wei Ju, Ming Zhang, and Yizhou Sun. Rigun: A rationale perspective for semi-supervised open-world graph classification. Transactions on Machine Learning Research, 2023b.
Jiaqi Ma, Ziqiao Ma, Joyce Chai, and Qiaozhu Mei. Partition-based active learning for graph neural networks. Transactions on Machine Learning Research, 2023.
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construc- tion of internet portals with machine learning. Information Retricval, 3(2):127-163, 2000.
Amrit Nagarajan and Anand Raghunathan. Fastrain-gun: Fast and accurate self-training for graph neural networks. Transactions on Machine Learning Research, 2023.
Nihal V Nayak and Stephen H Bach. Zero-shot learning with common sense knowledge graphs. Transactions on Machine Learning Research, 2022.
Shirui Pan, Jia Wu, Xingquan Zhu, Chengqi Zhang, and Yang Wang. Tri-party deep network representation. Network, 11(9):12, 2016.
Lucas Prieto, Jeroen Den Boef, Paul Groth, and Joran Cornelisse. Parameter efficient node classification on homophilic graphs. Transactions on Machine Learning Research, 2023.
Bernardino Romera-Paredes and Philip Torr. An embarrassingly simple approach to zero-shot learning. In International conference on machine learning, pp. 2152 2161. PMLR, 2015.
Edgar Schonfeld, Sayna Ebrahimi, Samarth Sinha, Trevor Darrell, and Zeynep Akata. Generalizod zero-and few-shot learning via aligned variational autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8247-8255 8255, 2019.
16

---

Published in Transactions on Machine Learning Research (10/2023)
Petar Veličkovié, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR. 2017.
Ziyu Wan, Dongdong Chen, Yan Li, Xingguang Yan, Junge Zhang, Yizhou Yu, and Jing Liao. Transductive zero-shot learning with visual structure constraint. Adcances in Neural Information Processing Systems, 32, 2019.
Chaoqun Wang, Shaobo Min, Xuejin Chen, Xiaoyan Sun, and Houqiang Li. Dual progressive prototype network for generalized zero-shot learning. Advonces Neural Information Processing Systems, 34:2936- 2948, 2021a.
Kuansan Wang, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. Mi- crosoft academic graph: When experts are not enough. Quantitative Science Studies, 1(1):396 413, 2020.
Xiaolong Wang, Yufei Ye, and Abhinav Gupta. Zero-shot recognition via semantic embeddings and knowledge graphs. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2018.
Zheng Wang, Jialong Wang, Yuchen Guo, and Zhiguo Gong. Zero-shot node classification with decomposed graph protolype network. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1769-1778 2021b.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non- con-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3733 3742, 2018.
Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata. Feature generating networks for zero-shot learning. In Proceedings of the IEEE conference on computer vision and pottern recognition, pp. 5542-5551, 2018.
Yongqin Xian, Saurabh Sharma, Bernt Schiele, and Zeynep Akata. f-vaegan-d2: A feature generating framework for any-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10275-10284, 2019.
Siyu Yi, Wei Ju, Yifang Qin, Xiao Luo, Luchen Liu, Yongdao Zhou, and Ming Zhang, Redundancy-fres self- supervised relational learning for graph clustering. IEEE Transactions on Neural Networks and Learning Systems, 2023.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph rastivve learning with augmentations. Advances in Neural Information Processing Systems, 33:5812-5823, 2020.
Jingyang Yuan, Xiao Luo, Yifang Qin, Zhengyang Mao, Wei Ju, and Ming Zhang. Alex: Towards effective graph transfer learning with noisy labels. In Proceedings of the 31th ACM International Conference Multimedia, 2023.
Qianmu Yuan, Jianwen Chen, Huiying Zhao, Yaoqi Zhou, and Yuedong Yang. Structure-awaro protein- protein interaction site prediction using deep graph convolutional network. Bioinformaties,, 38(1):125 132, 2021.
Qin Yue, Jiye Liang, Junbiao Cui, and Liang Bai. Dual bidirectional graph convolutional networks for zero- shot node classification. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2408 2417, 2022.
Eloi Zablocki, Patrick Bordes, Laure Soulier, Benjamin Piwowarski, and Patrick Gallinari. Context-aware zero-shot learning for object recognition. In International Conference on Machine Learning, pp. 7292-737777 PMI.R, 2019.
Zhiqiang Zhong, Sergey Ivanov, and Jun Pang. Simplifying node classification on heterophilous graphs with compatible label propagation. Transactions on Machine Learning Research, 2022.
17


Reviews:
[{"review_id": "03cf4gfFqa", "reviewer": "Reviewer_CMeh", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- The writing is clear and the paper is easy to follow\n- The proposed method outperforms other competitive baselines\n\n**Weaknesses**\n- The scalability to large number of nodes and classes (e.g., millions or more) seems challenging\n- There discrepancy in the accuracy metrics for two major baselines (DGPN/DBiGCN). The numbers presented in this submission are different from the numbers presented in the DGPN and DBiGCN papers. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studies zero-shot node classification problem, where each class also comes with a semantic description vector. The author proposed GraphCEN, which first forms the node-to-label joint information matrix, and then performs  node and class level contrastive learning based on the rows/columns of the join information matrix. Empirically, their proposed GraphCEN outperforms other state-of-the-art zero-shot classification methods. The author also conducted several ablation studies to justify the effectiveness of each components in GrpahCEN.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Technical Details**\n- Q1: In Algorithm 1, why constructing the class affinity graph `G_A` inside the while loop? If `G_A` is just the kNN graph from the class semantic description vectors `s_k`, then `G_A` is not subject to change right?\n- Q2: It is not clear how the model parameter `\\theta` is used in Sec 3.4.1 and Sec 3.4.2. Is it the MLP parameters to produce $H^a H^b, Z^a, Z^b$ from $H$ in Equation (4) ?\n- Q3: For time complexity analysis in Section 3.6, why $d^{'}$ is not used in the BigO notation of computing the joint information matrix? To compute $\\text{ReLU}(A^{hat}XW)$, we need to first multiply the node feature matrix $X \\in \\mathbb{R}^{N \\times d}$ with the weight matrix $\\mathbb{R}^{d \\times d'}$, which takes $O(N \\cdot d \\cdot d^{'})$?\n\n**Main Experiment Results**\n- Q1: There are some discrepancy between the numbers of `DGPN` in Table 2 of this submission versus the original numbers in Table 3 of DGPN paper. In this submission, the Class Split II accuracy of `DGPN` are 59.38 and 60.12 for Citeseer/C-M10M, respectively. However, in the original paper, the numbers are 61.90 and 62.46 for Citeseer/C-M10M, respectively. Any explanation?  \n- Q2: There are **large** discrepancy between the numbers of `DBiGCN` in Table 2 of this submission versus the original numbers in Table 3 of DBiGCN paper. In this submission, the Class Split II accuracy of `DBiGCN` are 58.29 and 66.23 for Citeseer/C-M10M, respectively. However, in the original paper, the numbers are 60.11 and 71.46 for Citeseer/C-M10M, respectively. Any explaination?\n\n**Ablation Studies**\n- Q1: How important is the entropy regularization term in Equation(8)? Any ablation results on that?\n- Q2: How scalable is the proposed method? Any results on larger scale datasets (e.g., million numbers of nodes)?", "broader_impact_concerns": "To my knowledge, there's no concerns regarding the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qygC3pkZyl", "reviewer": "Reviewer_GSsH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "At a high level the work in this paper clearly meets the requirements for acceptance criteria in TMLR. The model is supported by ample empirical evidence and ablation studies, and people in the community will certainly find the results interesting.\n\nThe writing of the paper leaves much to be desired, however. There is a long list of typos which are substantial enough to make some parts of the paper hard to parse. More problematic, however, is that certain parts of the model itself were not presented completely or clearly, to the point where I have no confidence that I would be able to reproduce the author's proposed architecture from reading the paper alone. In addition, to the extent that I understood it, certain parts of the proposed model seem suboptimal, and some explanations in the ablation studies are confusing. I will go into more detail in the following section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work the authors present a GraphCEN, a model for zero-shot node classification whose architecture enables the use of contrastive training objectives on both nodes and classes. The main novel contribution is the incorporation of class affinity information, which is captured as an adjacency matrix with values related to the dot product of feature vectors for each class. This matrix multiplies the output of a graph neural network (which aggregates node embeddings) to create what the authors refer to as the \"joint information matrix\", presumably because the rows of this matrix (which provide a distribution over classes) can be viewed as node embeddings, while the columns (which provide a distribution over nodes) can be viewed as class embeddings. They leverage this interpretation to provide a contrastive learning objective, wherein slightly different joint information matrices are produced as a result of augmenting the underlying graph, and corresponding rows and/or columns are encouraged to be more similar to each other than others.\n\nThe authors run zero-shot node classification experiments on three datasets, demonstrating state-of-the-art performance on all three. They also include comprehensive ablations considering various aspects of the contrastive loss.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "\n1. Overall, there are a number of issues with the mathematical formulation, notation, and rigor in the paper. For example: when defining the class affinity graph (around equation (2)) the vertices are defined as $\\mathcal V_G = \\{v_1, \\ldots, v_{|\\mathcal C|}\\}$, but this notation was already used to describe the nodes of the original graph $G$. Thus, a strict interpretation of what was described would amount to the class affinity graph using nodes from the original graph as opposed to classes, which is what was presumably intended. One way to fix this would be to simply define $\\mathcal V_G = \\mathcal C$ (after all, that's the point of abstracting graphs away from the actual nodes they connect) however since we don't actually use the graph structure and rather simply use the adjacency matrix $\\mathbf A^s$ (which, incidentally, does not have entities in $\\{0,1\\}$ and therefore the graph should be qualified as a *weighted* graph) I think it would be far simpler to avoid the whole auxiliary graph definition entirely and simply state that you create a similarity matrix as described in equation (2). Furthermore, the superscript $s$ was already used in describing the dimension of the feature vectors for the classes, thus making $\\mathbf A^s$ look like a matrix $\\mathbf A$ raised to an integer power $s$. Instead, you presumably intended this superscript on $\\mathbf A$ to indicate a dependency on the feature vectors themselves, e.g. $\\mathbf A ^{\\mathbf S}$. This is fairly representative of the issues with mathematical notation throughout the paper, which leads to significant confusion in various points.\n2. Space does not seem to be at a premium here, and it would be worthwhile to expand on the technical details of the model so that the reader would actually be capable of reproducing the work directly. For example:\n    1. The paper would become more self-contained if a definition of the neighbors $\\mathcal N(s)$ was included near equation (2).\n    2. In section 3.4.1 it is unclear to me what is meant by \"After encoding from our well-designed GNN in Section 3.3 followed by a node head (i.e., multi-layer perceptron, MLP), we can obtain the augmented joint information matrices $\\mathbf H^a$ and $\\mathbf H^b$.\" Are you saying that $\\mathbf H^a = W_2\\operatorname{ReLU}(W_1 \\operatorname{ReLU}(\\hat{\\mathbf A^a} \\mathbf X \\mathbf W) \\mathbf A^s)$ and  $\\mathbf H^b = W_2\\operatorname{ReLU}(W_1 \\operatorname{ReLU}(\\hat{\\mathbf A^b} \\mathbf X \\mathbf W) \\mathbf A^s)$, where $\\hat{\\mathbf A^x} = R(\\operatorname{Aug}_x(\\mathbf A))$, $R(\\mathbf A) = \\widetilde {\\mathbf D}^{-\\frac 1 2} \\widetilde{\\mathbf A} \\mathbf{D}^{-\\frac 1 2}$ for some adjacency matrix $\\mathbf A$, where $\\widetilde{\\mathbf A} = \\mathbf A + \\mathbf I$ and $\\widetilde{\\mathbf D}$ is it's degree matrix?\n    3. After equation (8) it would be useful to define exactly the function $H(\\cdot)$ as opposed to just stating it is the entropy function, because $\\mathbf Z$ is not a random variable and so it is not clear to me how this is defined.\n4. In Section 4.1 some details of the datasets are provide, including the fact that nodes represent different publications and the edges represent citation relationships, however there is no information about what the classes are. This seems rather crucial, given the main goal of the paper.\n5. In Section 4.6 you analyze the differences between various choices of graph augmentation, and present results in Figure 3, however I could not find any information about what the horizontal or vertical axes represent. I assume one is for the augmentation applied to the nodes, and the other is the augmentation applied to the classes when performing the contrastive loss. When doing so, one could either (a) perform the ablation on the original graph and then create the embeddings, or (b) apply the ablation only for the part of the model which utilizes the adjacency matrix (eg. if the method \"ED\" is used for the nodes, it could be used to generate a new matrix $\\mathbf A$ which is then used for both $\\hat {\\mathbf A}$ and $\\mathbf A^s$ when creating the augmented node embeddings, or it could be used for just $\\mathbf A$). In writing this I recognize that the answer may also be \"neither\", particularly since the resulting Figure 3 does not have the elements in the upper-right (presumably because they would be symmetric, which is not the case with my two proposed interpretations). If I missed a description of this I apologize, however I think the difficulty in understanding here further highlights the benefit of using clear notation and writing explicitly the function which ultimately computes the embeddings in these ablation settings as well. Perhaps, instead of either my interpretations, this analysis considers the augmented graphs used to produce $\\mathbf H^a$ and $\\mathbf H^b$, but in this case it is very unclear to me why using two augmented graphs would ever be better than using the original graph in some capacity.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AoKjbrKN0Q", "reviewer": "Reviewer_rL7A", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Discussion:**\n\nThe authors suggest a contrastive learning method that acts on an embedding matrix $Z$ of dimensions $n \\times c$ where $n$ is the number of nodes, and $c$, the number of classes (including unseen ones). The method combines _two_ contrastive learning objectives - one acting along the rows of $Z$ (node-wise_ and the other along the columns of $Z$ (class-wise). \n\nThe node-wise contrastive learning simply considers two views of the original graph—-generated in the usual kind of way by dropping edges at random and applying a diffusion operator to the adjacency matrix. The class-wise loss acts similarly but on the columns. Column $i$ of the embedding of both views are trained to be the same, and all other pairs of columns are treated as negative pairs. One way to think of this approach is as normal (graph) contrastive learning, but instead of arbitrary node embedding dimension, the dimension _has to be $c$_, the number of classes, and the column-wise loss forces each dimension of vector to be decorrelated from the others (quite similar to Barlow Twins in this way).\n\nFinally, to actually force the trained-to-be-decorrelated embedding dimensions to actually correspond to particular classes, the authors introduce a third loss - a standard cross-entropy loss on the classes that are seen during training (recall, the setting assumes only some of the classes are seen in training).\n\nEach of the three losses used in the method does seem to address different desired properties of the model and broadly make sense in combination.\n\n---\n\n**Conclusion:**\n\nThe problem setting is interesting enough, and the method broadly makes sense (see questions for concerns) and seems to produce decent results. I expect that the TMLR audience would find this work interesting. A caveat is that I am not abreast of the state-of-the-art in this problem setting, so am unable to assess the similarities to prior work. \n\nHowever, I have a couple of concerns that relate to critical parts of the framework. I have laid out these concerns under the “questions” section.   Overall I am likely trending toward accepting this paper. However to clearly register my concern with these questions, and the need for a clear answer to them, I have currently assigned a weak reject score. This will most likely rise after hearing the author’s response. \n\n\n---\n\n**Writing:**\nIn terms of writing clarity, I would like to be firm. I would like the authors to know that the spirit of this message is to improve the accessibility of their work to readers, and does not form a part of my assessment of the quality of the work, which determines my advocation for acceptance or rejection. \n\nThe writing needs considerable revision. Primarily I am concerned that sections 2 and 3 (setup and method) are organized poorly, making the ideas harder to digest than need be. Specifically, the information is either presented in an incomplete, or non-linear form. Some examples of what I mean by this:  \n\n**Incomplete:**\n- Diffusion Equation 1 - you neglect to actually mention what the diffusion matrix $A’$ is used for. I am assuming that the adjacency matrix is simply updated to be this new diffused matrix. But if this is actually the case you should explicitly say so and do not assume the reader’s familiarity with what people use this diffusion for in graph contrastive learning.\n\n- The class affinity graph (sec 3.2) is never properly defined. Specifically, the definition of the neighborhood N(s) of a class is never defined, and a paper from 2021 is simply cited and readers are directed to read it. What makes this worse is that this 2021 paper does not use the words “class affinity graph” at all, and does not use the notation N(s) at all, making the reader have to dig through their nomenclature to find the relevant definition. \n\n**Non-linear**\n- Section 3.2 again. The “class affinity graph” is (partially) defined, but the reader does not yet know what it is for. It is only in the next section that its use in this context is revealed. This form of writing is common in mathematics, but machine learning audiences likely would like to know the big-picture of why this math is being introduced before it is introduced. \n\nThe authors also are inconsistent in notation. $H$ and $Z$ denote the same matrix in different places (one when doing row-wise, and the other when doing column-wise loss). I see the reasons for a switch-up, but make the transition clearer. Related to this, there are often vague or unclear phrases used, including at key moments when the authors are trying to establish the motivation for their work.\n\nFor example, “Inability to explicitly model the dependencies between nodes and classes” (Key limitation ii). The authors should try to make this much more specific since I don’t know what this means in its current form. I suspect the idea is that the model should be using label information _as part of its layer-wise updates_ - i.e., do not just use class information on output embedding vectors via cross-entropy loss etc., but actually use the labels to determine information propagation through the model. Finally, although this sounds a sensible enough sentiment, it seems generic and not directly related to the question of zero-shot node classification (see questions for more on this). ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": " First, thank you to the authors for an interesting piece of work. I enjoyed reading the ideas, results, and various connections it all points to. \n\n**Brief summary:**\n\nThe paper addresses the problem of zero-shot node classification in graphs, which involves predicting the classes not seen during training. The authors propose a method that combines contrastive learning and supervised learning, using the unsupervised component to enforce a consistent structure/relation between the seen and unseen classes, thereby working towards that stated goal of generalizing to unseen classes. \n", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Questions:**\n\n- What is the effect of $A^s$? This is included as part of the model architecture but is orthogonal to the contrastive learning loss proposed. How does your method perform without this $A^s$? If the method is only beating baselines due to this design component that could equally be added to other baseline methods then that’s a concern. A response to this point is an absolute must, as right now I am concerned that the experimental setting is unfair on the baselines and need reassurance that this is not the case. Or is the use of $A^s$ in this way a novel contribution? If this is so, an explanation of how this paper’s use of $A^s$ differs from the original Wang et al. 2021 would be appreciated. \n\n- Why have you added the entropy $H(Z)$ to your contrastive loss “to prevent collapsing into trivial outputs of the same class”? I would be grateful for some clarification of this component, since to my eyes it seems superfluous - the negative samples are there precisely to avoid this collapse, so why the need for this term? Can this term also be removed from method safely without hurting performance?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper aims to improve zero-shot node classifications in graphs. In this paper, the authors present a framework which first constructs an affinity graph to model the relations between the classes and then a node- and class-level contrastive learning are used to jointly learn node embeddings and class assignments in an end-to-end manner. However presentation and writing of the paper leaves more to be desired. We thank the authors and reviewers to actively engage in discussion for making the paper better. During discussion many of the reviewer concerns/questions were resolved like the one about discrepancy in the accuracy metrics or role of $A^*$. However some concerns remain about the presentation and writing. Nevertheless, the proposed method and technique are correct, experimental result demonstrate robustness to quantization in one way, and the approach will be of interest to the community, hence I propose to accept the paper with some minor modifications. Please improve the presentation and writing as pointed out by the reviewers. Note that errors pointed out by were not exhaustive, and there may still be other areas for improving the writing.
Decision: Accept (poster)