SpeAr: A Spectral Approach for Zero-Shot Node Classification
Authors: Ting Guo
Da Wang
Jiye Liang
Kaihan Zhang
Jianchao Zeng
Venue: NeurIPS 2024 poster (2024)
Keywords: zero-shot node classification
graph neural networks
class prototypes
spectral contrastive loss
Primary area: graph_neural_networks

Abstract: Zero-shot node classification is a vital task in the field of graph data processing, aiming to identify nodes of classes unseen during the training process. Prediction bias is one of the primary challenges in zero-shot node classification, referring to the model's propensity to misclassify nodes of unseen classes as seen classes. However, most methods introduce external knowledge to mitigate the bias, inadequately leveraging the inherent cluster information within the unlabeled nodes. To address this issue, we employ spectral analysis coupled with learnable class prototypes to discover the implicit cluster structures within the graph, providing a more comprehensive understanding of classes. In this paper, we propose a spectral approach for zero-shot node classification (SpeAr). Specifically, we establish an approximate relationship between minimizing the spectral contrastive loss and performing spectral decomposition on the graph, thereby enabling effective node characterization through loss minimization. Subsequently, the class prototypes are iteratively refined based on the learned node representations, initialized with the semantic vectors. Finally, extensive experiments verify the effectiveness of the SpeAr, which can further alleviate the bias problem.

SpeAr: A Spectral Approach for Zero-Shot Node Classification
Ting Guo', Da Wang", Jiye Liang"; Kaihan Zhang', Jianchao Zeng'
1. Data Science and Technology. North University of China, Taiyuan, China.
2. School of Computer and Information Technology, Shanxi University, Taiyuan, China.
Abstract
Zero-shot node classification is vital task in the field of graph data processing, aiming to identify nodes of classes unseen during the training process, Prediction bias is one of the primary challenges in zero-shot node classification, referring to the model's propensity to misclassify nodes of unseen classes as seen classes. However, most methods introduce external knowledge to mitigate the bias, inadequately leveraging the inherent cluster information within the unlabeled nodes. To address this issue, we employ spectral analysis coupled with learnable class prototypes to discover the implicit cluster structures within the graph, providing more comprehensive understanding of classes. this paper, we propose a Spectral Approach for zero-shot node classification (SpeAr). Specifically, we establish an approximate relationship between minimizing the spectral contrastive loss and performing spectral decomposition the graph, thereby enabling effective node characterization through loss minimization. Subsequently, the class prototypes are iteratively refined based on the learned node representations, initialized with the semantic vectors. Finally, extensive experiments verify the effectiveness the SpeAr, which can further alleviate the bias problem.
Introduction
Graph data is widely used to reveal interactions between various entities, such as citation networks [. social networks [2], recommendation systems (3), etc. In graph data structures, various entities are abstractly represented through the form of nodes, while their complex relationships are precisely depicted through the connections of edges. Node classification analysis, focuses predicting labels for unlabeled nodes by harnessing the relational structure of the graph and a select subset of labeled nodes. The advent of graph neural network (GNN) technology, as delineated in [6] 8), represents a monumental stride in the domain of node classification. This development has accelerated advancements in the analysis of graph-structur data.
[ an essential task in graph data
Nevertheless, the ongoing dynamics of real-world networks, characterized by the continual emergence of new nodes and edges, facilitate the discovery of new classes, thereby introducing a series of innovative challenges for node classification tasks [9] Traditional GNN-based node classification models struggle to handle the identification of new classes in dynamic and open environments. For instance, citation networks continue to grow, the emergence of new research papers promotes the rise of new academic fields. A key challenge the effective integration of newly emerging nodes into the pre-existing foundational classes nascent ones currently in development. Therefore,
"Corrssponding author. Email: ljy @xx..eduunnn This work is supported by the National Science and Technology Major Project (2000AAA1061022) the National Natural Science Foundation of China (62376141), the Natural Science Foundation of Shanxi Province, China (2222))001222277))
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

constructing a model that can recognize nodes of unseen classes offers significant practical utility for the analysis of graph data.
Zero-shot node classification (ZNC) [0] presents a potent strategy for tackling the recognition of unseen classes that are absent from the labeled nodes. It endeavors to harness external categorical knowledge (e.g. semantic vectors) and comprehensive graph data (e.g. node attributes and structural information), to classify nodes belonging to unseen classes. In ZNC, prediction bias is significant issue, where models incorrectly predict unseen class nodes as seen classes, Several studies concentrate on constructing optimized node representations and ensuring their alignment with semantic vectors in the latent space. This alignment is achieved by minimizing the distance between nodes and their corresponding class semantic vectors. DGPN [IO] captures the local and global embedding of graph nodes, devising distance loss function that bridges the gap between graph representations at varying scales and semantic vectors. MVE [ underscores the insufficiency of modeling node features from single view in describing the comprehensive essence of node, thus advocating for multi-view approach to augment node features. In addition, the utilization of inter-class relationships is also crucial in ZNC. DBiGCN [2) incorporates class relationships into the node embedding process to facilitate the model's s transfer to unseen classes. GraphCEN [3] enhances model adaptation to unseen classes through dual contrastive loss mechanism that captures node-class dependencies.
Existing research predominantly leverages external knowledge, such as semantic vectors and the category relationships stemming from them, to mitigate prediction bias. However, most methods insufficiently exploit the cluster information inherent within unlabeled nodes. Strategically leveraging this latent information can reveal hidden class cluster structures, thereby facilitating a comprehensive understanding of the categories present within the graph. In this paper, we present a novel method, termed Spectral Approach for ZNC (SpeAr), that incorporates spectral analysis and learnable class prototypes to discover the intrinsic cluster structure embedded within the graph. Specifically, we introduce spectral contrastive loss for optimizing node embedding and posit that minimizing this loss is approximate to conducting a spectral decomposition of the graph. The spectral contrastive loss can elucidate the intrinsic structure of graph data, ensuring high degree of discriminability among classes in the latent space. In addition, we initiate the class prototypes by class semantic vectors and iteratively refine these prototypes based on the node embeddings.
In summary, our contribution is fhree-fold: (1) We propose a spectral contrastive loss to update nodes and establish an approximate relationship between the node embeddings obtained from this loss and the feature vectors derived from spectral decomposition.
(2) Taking class semantic vectors as the
initial class prototypes, we iterative refine these prototypes building upon the node embeddings. (3) Experiments demonstrate that our proposed method, SpeAr, can alleviate the bias issue that occurs in existing ZNC tasks, thereby enhancing the model's ability to recognize unseen classes.
2 Related Work
Zero-shot learning. Zero-shot learning [1] has gained attention for its potential for wide range of applications in fields such as image processing [15], speech recognition (16), and natural language processing 17.. In zero-shot learning models, commonly used external semantic knowledge includes attribute vectors and semantic vectors, among others. In particular, text vectors play an important role in representing textual content, and they translate words, sentences, and even whole paragraphs into vector representations through techniques such as Word2 (18), GloVe [1), and BERT 20 Existing methods focus on establishing mapping relationships between samples and external knowledge for effective knowledge transfer to unseen classes. These include direct mapping methods 21 22], which work by constructing a mapping function between samples and external knowledge and optimizing it; generation-based methods [15] 23), which train semantic-visual generators to generate samples for the unseen classes; and local smbedding-basee methods 24 22], which use the attributes to guide the learning and migration of discriminative local embeddings. While the aforementioned methods have exhibited noteworthy progress in managing regular data types like images, they encounter obstacles in handling ZNC. The distinct struetural attributes of graph data often hinder traditional zero-shot learning techniques from capturing and addressing these crucial properties, thereby rendering them ineffective in resolving the ZNC issue.
Node classification. Node classification represents a pivotal task within the domain of graph data analysis. The introduction of GNNs has catalyzed transformative evolution in the advancement of

---

node classification techniques 77. GNNs possess the capability to concurrently process the adjacency structures and feature information of nodes. A multitude of methods predicated on GNNs [26 27 have been introduced, progressively emerging as the predominant technologies for addressing node classification challenges. These conventional approaches are largely predicated on the assumption that the class labels present in the training data encompass all possible classes. However, the reality is that new categories continually emerge. Regrettably, existing GNN frameworks exhibit notable deficiencies when it comes to effectively handling unlabeled nodes from unseen classes.
3 A Spectral Approach for Zero-shot Node Classification
3.1 Problem Formalization
Let G = {V,E) denote graph. V the set of nodes with |V] = N, & is the set of edges. In the graph, the feature matrix is X RNxx is the dimension of features. A {0, 1JNNN denotes the adjacency matrix, where Gij = 1 if (u;, U2) E E, otherwise aij = 0.
For zero-shot node classification, C = (c...... Gcc is the class set, C, (ei, 9c,, } is the seen class set, C. {ei, "..,,...C (CI,...,cC... (C.)) is the unseen class set, C, U C.. = C and C. n C., G. Given an I E X, we use P to denote the marginal distributions of all data that x ~ P. and let P.. denote the distribution of labeled samples with class label c E C and P, denote those of unlabeled data. In addition, every class has a distinct class semantic vector 8; E Rd |C| class semantic vectors (CSVs) (Further details are provided in Section 4.1 can be formed into a matrix S E R Clxd, In graph G. there are N₂ labeled nodes from seen classes, forming the label matrix Y, E NR,., For ZNC, we predict labels for N,, unlabeled nodes from unseen classes, with predictions in Y, RC,,l Generalized zero-shot node classification (GZNC) predicts labels for Nu unlabeled nodes, including both seen and unseen classes, with predictions in Y... E R.Noe
For node embedding, we employ the GNN 7 model for node representations. Specifically, given the adjacency matrix A and the feature matrix X, aggregating information from neighboring nodes, the node embedding is:
=
(1)
where D' EA is the diagonal matrix of node degrees. W1 denotes the parameters of single-layer neural network, and a(-) is activation function. In addition to aggregating information about neighboring nodes, this study also emphasizes the importance of information about the nodes themselves in category mining. Therefore, we input each node into the neural network to obtain its representation, the embedding of nodes in the latent space is formulated as follows:
- A0 x/2 + +XWW),,
(2)
where W2 denotes the parameters of a single-layer neural network.
3.2
Preliminaries of Spectral Decomposition
Given a graph structure, we primarily employ spectral decomposition key technique to derive principled embeddings. Spectral decomposition, a robust mathematical instrument, has long exhibited its unparalleled value in clustering algorithms. Specifically, the spectral clustering methodology 28 29 30 utilizes spectral decomposition Ki 32 to learn the embeddings of samples in designated space, subsequently enabling efficient cluster analysis through algorithms like K K-means (28).
The spectral decomposition can be succinctly expressed as Á = QAQ", wherein Á D-1/2AD-1 Q denotes an orthogonal matrix encapsulating the complete set of eigenvectors of Å. A signifies a diagonal matrix whose diagonal entries are constituted by the eigenvalues of Á, corre- sponding to the eigenvectors in Q. Let A2, Ae denote the top-k eigenvalues, and qı, 42. Ik represent the corresponding top-k eigenvectors. We define F = [q), [ matrix of eigenvector moments, which serves novel, condensed representation of the sample. These eigenvectors embody the most significant directions within the data. Let 2; be the ith row of the matrix F* It turns out that 2 can serve as desirable node embeddings of 24.
92. ..., qu) E N x as the
3

---

3.3 Spectral Zero-shot Node Representation Learning
Spectral decomposition, by leveraging the relationships between data points, effectively unearths the implicit class cluster structures within graph, thereby obtaining node representations with class discriminability. One of the fundamental challenges of ZNC is identifying unlabeled nodes of unseen categories. These unlabeled nodes contain cluster information, and utilizing the information important for enhancing the model's recognition and understanding of unseen categories. By revealing the latent class cluster structures in the graph through spectral decomposition, we offer novel perspective for the ZNC.
It is imperative consider not only the relationships between nodes within the unlabeled set but also labeled information
seen classes. On this basis, we reconfigure the adjacency matrix by integrating the labeled information, so that A = a + 3A, where A' represents the label relation matrix. We use Agr' to denote the entries of the reshaped A, where Agr' = 0 represents that x and x' neither belong the same category have adjacency relationship. Otherwise, Agr' indicates that least one of the two conditions is met.
Aga' Σ GEC
EaNPe PP A'rx + 3E Aga'
(3)
and thus, we have A = Ex'ex Aasi. In this section, based on the reshaped adjacency matrix A and its normalization Å, refer to the Eckart- Eckart-Young-- Young- Mirsky theorem 333. where the loss of solving F can be formalized as:
min Lad (F, 1| FF ||r.
FERNXE
Now, we view f.T of F scaled version of learned feature embedding f: X R* Lad can be formulated as a variant of the contrastive learning objective, enabling us to theoretically establish the approximation between the learned node representations and the top-k singular vectors of Á. We formalize the approximation in Theorem 3.1
Theorem 3.1. We define fe ff (x) for some function f. a, are hyper-parameters. Then minimizing the loss function (.a(F, A) equivalent to minimizing the following loss function for which we term spectral contrastive loss,
Csalf) 2 -2aLi(()) - 20060))-9000+2222))))020222221 (s))).
(5)
where
Ci(f) Σ ENNPe A (()) f(xt),
Am
GEC
C2(f) = (er so f(x),,
Ca( = Σ Σ Eyp Elaal (() ((=)))
ELEC EyE
Ca(f) Σ EuwP, x E( E(''' (f(=x) f<=)))((>>)))))))))))))))))))) ))
CEEC
Cs(f) = EINP, -- - Ela'l IA, (ff(x) {
Proof. (sketch) We can expand Lea( F,, A) and obtain
Aur' - f.
Lad (FA)=
Σ
VAnAW
a..d EX
const +
Σ
(-2A (x) fxx) A.A. (xd)))
(6)
The first term is constant. The form of Leal (f) is derived from plugging A and A..

---

Analysis of Theorem 3.1, The L1 loss is tailored to reduce the distances between embeddings of label nodes sharing identical class labels, thereby enhancing intra-class compactness. The C2 loss targets the unlabeled node pair with the highest adjacency probability as positive pair, amplifying the discriminative capacity of the embedding space. Conversely, the L3. L., and Ls losses are dedicated to the strategic dispersion of embeddings associated with negative pairs. The L3 loss function is calibrated to induce a distinct separation between embeddings of labeled nodes with different class labels. The C4 loss treats labeled nodes and unlabeled nodes counterparts negative pairs. Finally, the Ls, loss further refines the embedding space by considering all remaining unlabeled node pairs, apart from those identified as positive in the C2 loss, negative pairs. Above all, these intricately designed loss (sci()) (f) facilitates the aggregation of similar nodes and the segregation of dissimilar ones within the embedding space significantly enhancing the representational efficacy of the embeddings.
The Theorem 3.1 of this section is primarily adapted from Theorem 4.1 in (2) and Theorem 3.1 in 84 The main differences from the original theorems are reflected in the following two aspects: First, the types of data objects processed are different; the proposed method focuses graph data objects that inherently possess structural characteristics. Second, during the construction of the loss function, we define unlabeled node pair exhibiting the highest adjacency probability positive pair. These adjustments make the Theorem 3.1 more aligned with the specific context and requirements of the research presented in this paper.
3.4 Class Prototype Learning
The proposed method employs a meticulously crafted spectral contrastive loss function ensure the inter-class separability of the embedding vectors obtained in the latent space. To achieve classification of unseen class nodes, we utilize valuable pseudo-label information to perform iterative updates on class prototypes, which serve as the centers for each class. These prototypes are initially instantiated with semantic vectors S that are rich in categorical information. Utilizing these vectors, we perform preliminary classification predictions for the nodes, thereby creating pseudo-labels. Based on these useudo-labels, the unlabeled nodes with pseudo-labes predictions exceeding preset threshold g are selscted to update unseen class prototypes. For labeled nodes with pseudo-label predictions matching the ground truth label, we use them to refine prototypes for seen classes. Our update rule is defined as:
Pa + p. 2 i = c and Pr(j) > q, D is the pseudo-label a (1 - ()) .P Pe
Pe
otherwise.
Here, we use Pr(j) > q to denote the selected nodes whose pseudo-label predicted probability exceeds threshold q- f is the updated parameter. This strategy not only significantly enhances the model's generalization capability for unseen classes but also ensures that the classification accuracy for seen classes is maintained and enhanced.
3.5 Training and Testing
Considering the significant modal differences between the initial class prototypes (i.e., semantic vectors) and node embedding, we design a carefully planned two-stage model training approach. This strategy aims to gradually narrow the inter-modal gap through a careful optimization process, leading improved and refined class prototypes. In the first phase, we focus on roughly tuning the prototype to establish solid starting point. Subsequently. in the second phase, labeling and adjacency- refinement of the prototypes ensures the quality of prototype learning. Through this staged training. our model can achieve a more accurate representation of the class prototypes.
In the first phase, the backbone is pre-trained using unsupervised spectral contrastive loss Lusat- Luset means that the positive samples are the nodes themselves, and the negative samples are selected from other nodes in the graph. In the second phase, the model is trained using Cuet-
During the test phase, the embedding z; of x; the latent space is obtained by the network. Based on the learned class prototypes, we predict its label by:
c² = arg max (z X PP). CEC./C
(8)
E CC corresponds to ZNC/GZNC tasks.
5

---

Stage 1
Stage 2
Lusel
N
update
initiate
Lisct
S
S2
update
WIKIFEDIA
initiate
class numnes
CSVs
class prototypes
class prototypes
A Tuned
Frozen
L unsupervised spectral contrastive loss
le spectral contrastive loss
Figure I: The overall framework for SpeAr. The whole training process consists of two stages.
4 Experiment
4.1 Experimental Setup
we seek to substantiate the validity of our proposed SpeAr through its application to three public datasets: Cora [. Citeseer M1 C-MI0M 35 To ensure equitable comparison, the data partitioning strategy mirrors that of the aforementioned methods. The dataset details are shown Table D For Class Split I and II, the evaluation criterion is classification accuracy. In addition to that, we also give the Class Split III to validate the effectiveness of SpeAr handling GZNC. The evaluation criterion is H, defined as H 2 x (seen unseen)/ (seen + unseen). The seen and unseen the classification accuracies of seen and unseen classes, respectively.
Datasets. Following the methods DGPN [10]. DBiGCN [1), and GraphCEN B3.
For class semantic vectors (CSVs), DGPN has delineated two primary categories: label-based CSVs, which are word embeddings derived from class names, and text-based CSVs, which are document embeddings textual descriptions related to the class. These vectors are extracted utilizing the esteemed natural language processing model, Bert-Tiny (36). In our experiments, we mainly use baxe- CSVs because text contains richer information. Furthermore, we examine the different impacts of employing distinct CSVs on the experimental outcomes.
Baselines Methods. DGPN draws upon suite of comparative methods rooted in traditional zero- shot learning. We follow DGPN and list algorithms such as DAP [37), ESZSL (21). ZS-GCN (38), WDVSc 39 Hyperbolie-ZSL 40). DAP and ZS-GCN, as well as their CNN-enhanced counterparts, DAP(CNN) and ZS- C.SGGC These conventional methods, predominantly tailored for visual domain data (e.g., image data) prediction are often ill-equipped to handle the graph data, thereby exhibiting limitations in effectively addressing the ZNC problem. Our study places a particular emphasis on evaluating the performance of our proposed SpeAr model against these established methods such as DGPN [0), DBiGCN 12], GraphCEN 13, underscoring the superior effectiveness of SpeAr in the context of ZNC.
Parameter Settings. In SpeAr, we employ a two-stage training strategy. During the first phase, the model is trained by the loss Luset, which is governed by parameters a and 3. In this stage, samples are utilized for updating category prototypes only if the predicted probability of pseudo- labels exceeds a predefined threshold q, with g restricted to the range {0.5, 0.6, 0.7, 0.8, 0.9). To ensure the stability of the update process, the prototype update parameter y is cautiously set 0.1 to prevent the introduction of erroneous information that might arise from higher values. In the second phase, the loss function Last continues with the parameters a and 3 established in the first phase. Here, we refine the prototype update process by selecting the top-s nodes with the highest pseudo-label probabilities, with s uniformly set to 1000 across all datasets. Furthermore, to achieve fine-tuning of the prototypes, the update parameter is further reduced to 0.01 in this phase.

---

Table 1: Summary of the datasets.
ZNC
GZNC
Dataset
Nodes
Edges
Features
Classes
Class Split
Class Split II
Class Split
[Train/Val//esst]
[TTrain/Vall Test//
[Train/Val/Test
Cora
2708
5429
1433
[3/0/4]
[2/2/3]
[3/0/1
Citeseer
3327
4732
3703
[2/0/4]
[2/2/3]
[2/0/6]
C-M10M
4464
5804
128
[3/0/3]
[2/2/3]
[3/0/6]
4.2
Experimental Results
Table 2 records the ZNC accuracies corresponding to Class Split and II. A thorough analysis of
Table yields the following insights:
a) For Class Split SpeAr
achieves notable accuracy en- hancement on the Cora, Cite- seer, and C-MIOM datasets when compared to existing methods. This enhancement underscores the potency of SpeAr's ss spectral decomposition and prototype up- dating tactics. By delving deep- er into the intrinsic class cluster structures embedded within un- labeled data, SpeAr significantly amplifies its comprehension and identification of unseen classes, while ensuring good separability between different classes.
Table 2: Zero-shot node classification accuracy (%).
Cora
Citeseer
C-M10M
RandomGuess
25.35
24,86
33.21
DAP
26.56
34.01
38.71
DAP (CNN)
27.80
30.45
32.97
ESZSL
27.35
30.32
37.00
Class Iplit
ZS-GCN
25.73
28.62
37.89
ZS-GCN (CNN)
16.01
21.18
36.44
WDVSc
30.62
23.46
38.12
Hyperbolic- -SL
26.36
34.18
35.80
DGPN
33.78
38.02
41.98
DBiGCN
45.14
40.97
45.45
GraphCEN
48.43
40.77
44.17
SpeAr (Ours)
60.48
59.72
54.22
(b) For Class Split II, SpeAr shows a significant advantage over the state-of-the-an methods. In this scenario, model parame- ters determined using the val- idation set. This approach is jus-
Improve
24.88%
45.77%
19.30%
RandomGuess
32.69
50.48
49.73
DAP
30.22
53.30
46.79
DAP (CNN)
29.83
50.07
46.29
ESZSL
38.82
55.32
56.07
Class Sp
ZS-GCN
29.53
52.22
55.28
tified within the methodology of our study, as our objective is to thoroughly explore the informa- tion across all classes. We be-
ZS-GCN (CNN)
33.20
49.27
51.37
WDVSc
34.13
52.70
46.26
Hyperbolic-ZSL
37.02
46.27
55.07
DGPN
46.40
61.90
62.46
lieve that a model optimized ing validation sets can achieve a comprehensive understanding of the classes and is applicable to some extent the recognition of unseen classes.
DBiGCN
49.20
60.11
71.86
GraphCEN
50.61
60.47
70.83
SpeAr (Ours)
58.20
75.13
79.00
Improve
15.00%
21.37%
9.94%
Table 3 records the GZNC accu-
racies corresponding to Class Splits III. A thorough analysis of 3able the following insights: We record the performance of the DGPN and DBiGCN addressing the GZNC problem. Table 3 shows that both DGPN and DBiGCN exhibit suboptimal performance when tackling GZNC. The clas- sification accuracies for unseen classes are less than satisfactory and there is significant prediction bias. In contrast, SpeAr concurrently learns the prototypes for both seen and unseen classes during its training process. Experimental results indicate that SpeAr achieves significant enhancement in classification accuracy of unseen classes compared to DGPN and DBiGCN, and gained better H value. This enhancement indicates that SpeAr bolsters model's comprehension of unseen classes through more intensive mining of the clustering information embedded within unlabeled nodes, thereby enhancing inter-class separability and partially mitigating prediction bias. However, we also note that despite the progress made by SpeAr, there is still room for improving its performance.
7

---

Table 3: Generalized zero-shot node classification accuracy (%)
Cora
Citeseer unseen
C-MIOM
seen
unseen
seen
H
seen
unseen
DGPN
94.26
0
89.76
0
96.71
DBiGCN
66.39
1.27
2.50
35.70
4.83
8.50
30.04
4.30
7.52
SpcAr(Ours)
34.84
26.32
29.98
27.17
25.78
26.45
26.75
18.95
22.19
Modell Model2
Clscificatton Accuracy
80
catton
70-
20
Ciowr tert CR0DM hol.
100
0.5
0.6
0.7 0.8
0.9
The values of
(a) Moddl ablation under Class Split
(b) Anxlysis threshold value
Figure 2: Ablation study and parametric analysis of threshold value g.
Future work will further explore how to optimize the model achieve superior GZNC classification results.
4.3 Ablation Study
In SpeAr, the loss Lae plays essential role, as the node embeddings optimized by it approximate eigenvectors obtained through spectral decomposition. By optimizing the Laet, we not only capture the local connectivity information of the graph but also uncover the implicit class cluster structures within the unlabeled nodes, thereby achieving comprehensive of the class information depicted in the graph. In Figure (a), the Modell indicates that upon the removal of the Leef loss from SpeAr, its performance experiences a significant degradation, especially on the Citeseer dataset. This result strongly demonstrates that the Lact, by effectively leveraging the relationships between nodes in the graph, deeply mines the categorical information on the graph and enhances the discriminability between classes.
Furthermore, the mechanism of prototype updating is equally crucial. Random initialization of prototypes can lead to instability in the results. Therefore, this study employs semantic vectors the initial values for prototypes and devises two-stage training strategy. In the first phase, we utilize an unsupervised spectral contrastive loss for node embedding. In the second phase, we take the updated prototypes from the first phase as the initial values and further optimize the representations of nodes and prototypes using supervised spectral contrastive loss. In Figure 2 (a), Model2 is the model that omits the first phase and directly uses semantic vectors as the initial prototypes for the second phase. The results decline across three datasets. This phenomenon further confirms the effectiveness of our designed two-stage training strategy.
4.4 Parametric Analysis
The threshold value q is instrumental in determining the subset of nodes that contribute to the prototype update process. When a sample's pseudo-label probability exceeds the threshold g. updates the class prototypes. The choice of q depends model and dataset: low q risks noisy pseudo-labels, degrading performance, high ensures accuracy but limits unlabeled data utilization. In SpeAr, select using the validation set. As shown in Figure (b), across three datasets, varies, the accuracy changes in both the test and validation sets are roughly consistent, confirming the strategy's effectiveness and reliability.
SpeAr, List includes two pivotal parameters a and 3, deciding the relative significance of labeled and unlabeled samples, respectively. As illustrated in the Figure 3 the selection range for and is constrained to the set (0.25, 0.5, 0.75, 1). We record the effects of a and 8 on Cora, Citeseer, and C-M10M. The and 3 regulate the contribution of labeled and unlabeled samples in overall loss. The

---

Classitiattin Accuraty
Classification Accuracy
Classiiiaatio Accuracy
60
55
55
80
50
45
40
35
1.00
/
0.50
0.25
1.00
0.75
0.50
0.25
1.00
0.75
0.50
0.25
0.75 0.50 0.25
1.00
0.25
1.00
0.75
0.50
0.25
Cora
Citeseer
C-MIOM
Figure 3: The effects of and 3 on Cora, Citeseer, and C-MIOM.
DGPN
DBIGCN
SpeAr (Ours)
Theory
Theory
Theory
Probaliilstic Methhds
Probabiletic Methods
Methods
Genetic Algorfthrr
convett ai Ale Asporte
Genetie Auserti
Case Based
Case Based
Case Based
Figure 4: An example of the SpeAr's effectiveness in mitigating prediction bias on Cora dataset.
results reveal notable trend: when the value of is larger than a, the model generally exhibits better performance, with this advantage being particularly prominent when 3 1. The conclusions meet our expectations. A larger 3 can reduce overfitting to seen classes and better exploit the clustering information embedded in unlabeled nodes. Thus, the parameters need to satisfy B > This suggests that the a and B are not sensitive to some extent.
4.5 Further Analysis.
T demonstrates the significant effectiveness of the SpeAr model in mitigating predictive bias, with marked improvements in classification accuracy for both seen and unseen classes. In addition, for Cora, Citeseer, and C-M10M, we find that existing models fail to recognize some of the categories. Initially, we define recall as the proportion of samples correctly classified into a class relative to the recall of certain classes is
the total number of samples in that class. As shown in Figure extremely low or even zero for DGPN and DBiGCN on Cora dataset. For instance, the unseen classes "Probabilistic Methods" and "Theory" of Cora exhibit zero recall rate when predicted by DGPN model. This reveals the prediction bias issue inherent in current methods when addressing the 7NC problem, where some classes are not correctly identified, and unseen class nodes are crroncously concentrated in a few incorrect classes. contrast, the SpeAr model provides more accurate classification outcomes for all unseen classes. These results further confirm that the SpeAr model can improve the recognition of unseen classes and enhance class separability by utilizing and mining the class cluster information in unlabeled nodes.
SpeAr has demonstrated exceptional performance in addressing the ZNC problem. We argue that this methodology is equally efficacious when applied to conventional zero-shot learning endeavors, which are predominantly concerned with the identification of unseen classes within Euclidean space. When applying SpeAr to such tasks, the central challenge lies in effectively constructing the adjacency relationships between samples and integrating them with the loss function proposed in this paper.
5 Conclusion
This paper introduces spectral method designed to address the challenge of 7SNC, ensuring the effective unearthing of class cluster structures within graphs and enhancing the separability

---

between classes. Unlike existing approaches that focus on leveraging external knowledge to mitigate predictive bias, our novel approach SpeAr accentuates the exploration of the inherent, implicit cluster information encapsulated within the data of unlabeled nodes. SpeAr optimizes node embeddings by minimizing spectral contrast loss and iteratively updates the class prototypes with semantic vectors as initialization. Empirical results demonstrate that SpeAr achieves significant accuracy improvements in tackling ZNC and GZNC problems, effectively alleviating predictive bias, future research, we plan to extend the concepts of this study to other tasks with similar training data, thereby further validating its universality and efficacy.
Acknowledgement
Research is supported by the National Science and Technology Major Project (2020AAA01061022) the National Natural Science Foundation of China (62376141). and the Natural Science Foundation Any opinions. findings, conclusions, or recommen-
of Shanxi Province, China (20220302122207) dations expressed in this material are those of the authors and do not necessarily reflect the views, policies, or endorsements either expressed or implied, of the sponsors.
References
[1] Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad. Collective classification in network data. Al magazine, 29(3):93-9)) 93, 2008.
[2] Ke Liang Yue Liu, Sihang Zhou, Wenxuan Tu, Yi Wen, Xihong Yang, Xiangjun Dong, and Xinwang Liu. Knowledge graph contrastive learning based relation-symmmetriall structure. IEEE Transactions on Knowledge and Data Engineering, 36(1 36(1):226-238 2023.
[3] Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong, and Qing He. survey knowledge graph-based recommender systems. IEEE Transactions on Knowledge and Data Engineering. 34(8):354933566 2020.
[4] Smriti Bhagat, Graham Cormode, and S Muthukrishnan. Node classification in social networks. Social network data analytics, pages 115-148, 2011.
[5] Qitian Wu, Wentao Zhao, Zenan Li, David P Wipf, and Junchi Yan. Nodeformer: A scalable graph structure learning transformer for node classification. Advances in Neural Information Processing Systems, 05:27787-27401, 2022.
[6] Joan Bruna, Wojeiech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and deep locally connected networks on graphs. In International Conference on Learning Representations, 2014.
[7] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations, 2016.
[8] Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, Fuchun Sun, and Kunlun He. A survey of knowledge graph reasoning graph types: Static, dynamic, and multi-modal. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1-20, 2024.
[9] Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118 13:22118-22133, 2020.
[10] Zheng Wang, Jialong Wang, Yuchen Guo, and Zhiguo Gong. Zero-shot node classification with decomposed graph prototype network. In Proceedings of the ACM SIGKDD conference on knowledge discovery & data mining, pages 1769-1779. ACM, 2021.
[11] Jiahui Wang, Likang Wu, Hongke Zhao, and Ning Jia. Multi-view enhanced zero-shot node classification. Information Processing & Management, 6036), 2023.
10

---

[12] Qin Yue, Jiye Liang, Junbiao Cui, and Liang Bai. Dual bidirectional graph convolutional networks for zero-shot node classification. In Proceedings of the ACM SIGKDD conference on knowledge discovery & data mining, pages 1448-2211 ACM, 2022.
[13] Wei Ju, Yifang Qin. Siyu Yi, Zhengyang Mao, Kangjie Zheng, Luchen Liu, Xiao Luo, and Ming Zhang. Zero-shot node classification with graph contrastive embedding network. Transactions on Machine Learning Research, 2023.
[14] Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly. IEEE transactions on pattern analysis and machine intelligence, 41(9):2251 -2655, 2018.
[15] Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata. Feature generating networks for zero-shot learning. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 5542-5551. IEEE, 2018.
[16] Edresson Casanova, Julian Weber, Christopher D Shulby, Arnaldo Candido Junior, Eren Gölge, and Moacir A Ponti. YourTTS: Towards zero-shot multi-speaker TTS and zero-shot voice conversion for everyone. In Proceedings of the international conference on machine learning. volume 162, pages 2709-2720. PMLR, 2022.
[17] Pratyay Banerjee and Chitta Baral. Self-supervised knowledge triplet learning for zero-shot question answering. In Proceedings the conference on empirical methods in natural language processing, pages 151-162, 2020.
[18] Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. In International Conference on Learning Representations, 2013.
[19] Jeffrey Pennington, Richard Socher, and Christopher D Manning, Glove: Global vectors for word representation. In Proceedings of the conference on empirical methods in natural language processing, pages 1532-1543, 2014.
[20] Vincent Micheli, Martin Martin d' H'Hoffschmidt, and François Fleuret. On the importance of pre-training data volume for compact language models. In Proceedingy of the conference empirical methods in natural language processing, pages 7853-7858, 2020.
[21] Bernardino Pareres and Philip Torr. An embarrassingly simple approach to zero-shot learning. In International conference on machine learning, pages 2152-2161. PMLR, 2015.
[22] Ziming Zhang and Venkatesh Saligrama. Zero-shot learning via semantic similarity embedding. In Proceedings of the IEEE international conference on computer vision, pages 4166-4174. IEEE, 2015.
[23] Hyeonwoo Yu and Beomhee Lee, Zero-shot learning via simultaneous generating and learning. In Advances in Neural Information Processing Systems, pages 46-56. MIT Press, 2019.
[24] Guo-Sen Xie, Li Liu, Fan Zhu, Fang Zhao, Zheng Zhang, Yazhou Yao, Jie Qin, and Ling Shao. Region graph embedding network for zero-shot learning. In Proceedings of the european conference on computer vision, pages 562-580. Springer, 2020.
[25] Dat Huynh and Ehsan Elhamifar. Fine-grained generalized zero-shot learning via dense attribute- based attention. Proceedings of the IEEE international conference on computer vision, pages 4482-4492 IEEE.
[26) Lecheng Kong, Jiarui Feng, Hao Liu, Dacheng Tao, Yixin Chen, and Muhan Zhang. Mag-gnn: Reinforcement learning boosted graph neural network. In Advances in Neural Information Processing Systems, volume 36, pages 12000-12021. MIT Press, 2023.
[27] Sitao Luan, Chenqing Hua, Minkai Xu, Qincheng Lu, Jiaqi Zhu, Xiao-Wen Chang, Jie Fu, Jure Leskovee, and Doina Precup. In Advances in Neural Information Processing Systems, volume 36, pages
MIT Press, 2023.

---

[28] Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. In Advances Neural Information Processing Systems, pages 849-856. MIT Press, 2001.
[29] Yiyou Sun, Zhenmei Shi, and Yixuan Li. A graph-theoretic framework for understanding open-world supervised learning. In Advances in Neural Information Processing Systems, volume 36, pages 23934 239677. MIT Press, 2023.
[30] Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self- supervised deep learning with spectral contrastive loss. In Advances in Neural Information Processing Systems, volume 34, pages 5000-5011, 2021.
[31] Canyi Lu, Jiashi Feng, Zhouchen Lin, Tao Mei, and Shuicheng Yan. Subspace clustering by block diagonal representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 4102):487-501, 2018
[32] Ery Arias-Castro, Gilad Lerman, and Teng Zhang. Spectral clustering based on local pea. The Journal of Machine Learning Research, 18(1):253-309 2017.
[33] Carl Eckart and Gale Young. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):1112118, 1936.
[34] Yiyou Sun, Zhenmei Shi, Yingyu Liang, and Yixuan Li. When and how does known class help discover unknown ones? provable understanding through spectral analysis. In Proceedings of the international conference on machine learning, volume 202, pages 33014-33043. PMLR, 2023.
[35] Shirui Pan, Jia Wu, Xingquan Zhu, Chengqi Zhang, and Yang Wang. Tri-party deep network representation. In International Joint Conference Artificial Intelligence, pages 1895-1901, 2016.
[36] Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Well-read students learn better: On the importance of pre-training compact models. arXiv preprint wwX.v::908.08900 2019.
[37] Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. Attribute- based classification for zero-shot visual object categorization. IEEE transactions on pattern analysis and machine intelligence, 36(3):453-465, 2013.
[38] Xiaolong Wang, Yufci Ye, and Abhinav Gupta. Zero-shot recognition via semantic embeddings and knowledge graphs. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6857-6866, 2018
[39] Ziyu Wan, Dongdong Chen, Yan Li, Xingguang Yan, Junge Zhang, Yizhou Yu, and Jing Liao. Transductive zero-shot learning with visual structure constraint. In Advances in Neural Information Processing Systems, pages 9972-9982. MIT Press, 2019.
[40] Shaoteng Liu, Jingjing Chen, Liangming Pan, Chong- Wah Ngo. Tat-Seng Chua, and Yu-Gang Jiang. Hyperbolic visual embedding learning for zero-shot recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 9270-9278. IEEE, 2020.
[41] Kuansan Wang. Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. Microsoft academic graph: When experts are not enough. Quantitative Science Studies, 1(1):396-413, 2020.
12

---

Appendix material
6.1 Proof of Theorem 3.1
Proof. We can expand (Fd A) and obtain
Lad (F))=
Σ
AUr'
-f
fg F.TFz)))
VAnAN
3. EX
const +
Σ
[x)
A.A (f(x) (
(9)
where f. = (z) a re-scaled version f()). At a high level, we follow the proof in [3],
while the specific form of loss varies with the different definitions of positive/negative pairs. The form of (f) is derived from plugging Agz' and Ag.
Recall that Agx' is defined by
Awa' Σ
E Pl Alex Pen Pupt' Amx'
(10)
thus have,
-2
E Azt f ((x) M(x)
2,I'EX
20
Σ
E Pa
p Aew f f(x')
f(x!)
6,E0
2a
Σ
EgnP Pe f(x') P f(x))
QEE
- 2a Σ
EgNP "
f(xt) - P Pftll h
f(x))
GEC
- -f) -
The penultimate equation is derived from the following lemma:
(sd(F) L(f) + const
where L(f) A -2 Er,x [f(x)T((xt))
f(xt)] + Er.--
[(xx))((T>))
(11)
Recall that A₃ is given by
Ag Σ Aga
(12)
x'EX
a Σ
EanPe Ax + BEan Ar.
(13)
13

---

thus plugging A₂ and Ag we have,
Σ Ax (11))
E BanPe As B.Boao Bi (a5 Bui Art + BB B. ) ((())
Σ
a
x,
GEC
=a²
E EanPe Ax Σ
BEWPE Ar (
2, E. GEC
eyEC
+ 20
Σ
BexaPey Ae Ar (f(x) 5 [ (x'))²
2
+ g2
Σ
BEWP, A,E Ar ff(=)
2,2'EX
=a² Σ
Σ
Pop "(''AA,, [[
44 4EC cjEC
+ 20:3
Σ
E E(r ((1
GEEC
+ 8 P
Edd'll,, (10
=a² = 2
The proof of Theorem 3.1 is finished.
6.2
Additional Experiments
6.2.1
Zero-shot node classification for large-scale data
For SpeAr, the spectral contrastive loss computes the similarities between samples, with a time complexity of ON N,,N + N,,,, + N.N; NuNN, ). Let N, be the count of labeled nodes, .N be the count of positive nodes of labeled nodes, and N the negative nodes. N,, is the count of unlabeled nodes, with N and N representing the count of positive and negative nodes, respectively. This complexity reveals a substantial demand for computational resources, presenting notable challenge for processing lcrge-scale graph data.
Following GraphCEN [3] we validate the efficacy of the SpeAr on large-scale dataset, such as ogbn-arxiv [4]] Ogbn-arxiv has 169343 nodes, and 2484941 edges. The feature dimension is 128 and the total class number is 40. Class split is [20/0/20], 20 seen classes as training set, 20 unseen classes testing set. Class split II is [13/1 3/14]/ 13 seen classes training set, 13 unseen classes as validation set, and 14 unseen classes as testing set. Confronted with memory limitations, we adopt multi-round subgraph extraction strategy. Specifically,
each round, we extract subgraphs that encompass both seen and unseen class nodes and execute the SpeAr algorithm on these subgraphs. Through this iterative process of extraction, we aim to progressively accumulate performance gains that mirror the execution of SpeAr on the entire graph, all while maintaining computational efficiency. As shown in Table 4 our proposed method SpeAr shows significant improvement in performance metrics compared to existing methods. The comparative analysis in the table highlights the superiority of our method in capturing class-disermminative information graph structures.
Table 4: A comparative performance analysis of DGPN, DBiGCN, and ours SpeAr for zero-shot node classification on ogbn-arxiv. (%)
DGPN
DBiGCN
GraphCEN
SpeAr(Ours)
Class Split I
22.37
21.40
23.96
30.45
Class Split II
21.95
25.92
28.36
32.20
14

---

Table 5: The Comparison of zero-shot node classification accuracy (%)
using the different CSDs.
Cora
Citeseer LABEL.
C-M10M
TEXT
LABEL
Decline rale
TEXT
Decline rate
TEXT
LABEL
Decline rate
DAP
26.56
25.34
-459 %
34.01
30.01
-1.76%%
38.71
32.67
11.60%%
ESZSL
27.35
25.79
-5.70%
30.32
28.52
-5.94%
37.00
35.02
-5.35%
ZS-GCN
25.73
23.73
-7.77%
28.62
26.11
-8.77%
37.89
33.32
-12.06%
WDVSe
30.62
18.73
-38.83%
23.46
19.70
-16.02%
38.12
30.82
-19.15%
Hyperbolie-ZSL
26.36
25.47
+3.38%
34.18
21.04
-38.44%
35.80
34.49
-3.66%
DGPN
33.78
32.55
-3.64%
31.83
-16.28%
-6.551%
DBIGCN
45.14
39.05
-13.49%
40.97
39.10
-3.10%
45.45
43.71
-3.83%
SpeAr(Ours)
60.48
49.52
-18.12%
59.72
48.88
-18.15%
54.22
47.05
-13.22%
DGPN Artificial Intelligence
DBIGCN Artificial Intelligence
SpeAr (Ours)
Artificial Intelligence
Human Compusn Interaction
Human Compa Computer Interaction
Human Compusee Interaction
Dalabase
Database
Patabasse
Machine Learning
Machine Leaming
Machine Leaming
Figure 5: An example of the SpeAr model's effectiveness in mitigating prediction bias Citeseer.
6.2.2 Discussion on Different CSVs
The impact of external knowledge from different sources on model outcomes significantly varied. In T we individually examined the effects of LABEL-based CSVs and TEXT-based CSVs as external knowledge. Given that TEXT data encapsulates a richer of categorical information, the SpeAr model utilizing text-based CSVs demonstrates superior performance. Indeed, when employing LABEL-based CSVs as the input external knowledge, SpeAr also outperforms existing methods, further corroborating the efficacy of the spectral contrastive loss and prototype updating mechanisms proposed in this paper for excavating and identifying categories on graphs. This series results underscore that approach significantly enhances the discriminability between different classes, thereby elevating the model's overall recognition capability.
6.2.3 Discussion on SpeAr model's effectiveness in mitigating prediction bias
We verify the benefits of SpeAr in mitigating prediction bias on the dataset Citeseer. As shown in Figure 5 the recall for certain classes is extremely low or even zero. For instance, the unseen classes "Human Computer Interaction" and "Artificial Intelligence" exhibit zero recall rate when predicted by the DBiGCN. In contrast, the SpeAr model provides more accurate classification outcomes for all unseen classes.
7 Limitation
Although the SpeAr model shows excellent performance on the ZNC task, its relatively high compu- tational complexity may become challenge when dealing with turge-scale graph data. Especially in application scenarios with limited resources or high real-time requirements, the high computa- tional cost may limit the usefulness of the model. Therefor, we effectively alleviate this problem by adopting the strategy of multi-round subgraph training. The model can gradually learn and integrate information from different subgraphs, thus realizing effective processing of large graph data while maintaining computational efficiency.
15

---

Experiments Compute Resources
Computation resources: We execute our code on computer with NVIDIA GeForce RTX 3090 (GPU) and Intel Xeon Gold 6254 (CPU).
9 Societal Impacts
The introduction of SpeAr has made significant contribution the advancement of zero-shot node classification tasks. It demonstrates tremendous potential in the field of data analysis, aiding researchers uncovering new insights and knowledge. There are no negative societal impacts on our work.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Both the abstract and introduction include the claims made in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations of the proposed method terms of time complexity, see Appendix 7
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Justification:
This paper provide the full set of assumptions and a complete (and correct)
proof. This paper proposes Theorem 3.1 Subsection 3.3 and complete proof in Appendix 6.1
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide details of the algorithm and its implementation in Section 3 and 4.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We provide the relevant code at github.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
moips.cc/pullic/guiddes /CodeSummidiSmii ssi iessonPoolin onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: This paper gives all the training and test details in Subsection 4.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: This paper reports the zero-shot node classification precision, generalized zero-shot classification results H in Subsection 4.2, and class recall for the Cora dataset in Subsection 4.5.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the experiments compute resources our work in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///wwurrr...cccccaaacccccaa
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both potential positive societal impacts and negative societal impacts of the work performed Appendix g
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
20

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the original paper that produced the code package.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcone com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
21

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Ye]]
Justification:
We provide details of new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
22


Reviews:
[{"review_id": "LVlegi2w2i", "reviewer": "Reviewer_rWPM", "summary": "This paper proposes a spectral approach for zero-shot node classification (ZNC) that addresses prediction bias based on node representation learning technique. It optimizes node representations by a two-stage training method with spectral contrastive loss and class prototypes, in which the class prototypes are initialized using document embeddings related to the class. Experiments have been conducted to verify the effectiveness of the proposed method on ZNC tasks.", "questions": "1. For large-scale datasets, subgraph-based training leads to slower convergence. The authors should discuss the training times compared to other methods across all datasets in details.\n2. Since the seen classes do not need to be classified during the test phase in the ZNC tasks, one of my concerns is that introducing too much seen class information during the train phase (e.g., reconfiguring the adjacency matrix and calculating spectral contrastive loss) may lead to overfitting and affect generalization to the unseen classes. Could the authors provide a discussion or explanation for how to address this issue?", "limitations": "The authors describe some limitations of the proposed method in Appendix.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. It is intuitive and reasonable to address the prediction bias in zero-shot node classification using the cluster information in the graph.\n2. Experiments have been conducted on three benchmarks and the results show the effectiveness of the proposed method compared with other ZNC method.\n3. The paper is well organized and explains the proposed method very clearly.", "weaknesses": "1. The design of the spectral contrastive loss function relies on previous work [29] with only slight variations in the specific form of positive and negative pairs.\n2. The proposed method might be inefficient on large-scale datasets since the spectral contrastive loss requires multiple loss calculations.\n3. It can be seen from Figure 2(b) and 2(c), the accuracy of the proposed method is sensitive to the hyperparameters $\\alpha$ and $\\beta$, which may require significant effort to make the method work on new datasets.\n4. The experimental settings are unclear. For the sake of reproducibility, the values of hyperparameters of the model (e.g., the embedding dimension of the graph neural network) should be specified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are immensely grateful to you for recognizing the reasonability and presentation of our work. Your valuable suggestions inspire us to improve our work further. \n\nIf you think the following response addresses your concerns, we would appreciate it if you could kindly consider raising the score.\n\n**> Replay to W1: Differences from the spectral contrastive loss in [29]**\n\nOur loss is designed based on the data and tasks, which is quite different from the loss in literature [29], as shown in the Table.\n\n| | Literature [29] | Ours |\n| - | -|-| \n| Data Format | Image Data | Graph Data | \n| Information Utilization | Unsupervised tasks, image augmentation | Label Information, nodes adjacency relationships | \n| Positive Pair Construction | Solely relying on augmented images as positive samples | (1) Relying on label information for labeled samples; (2) Relying on adjacency relationships for unlabeled samples| \n| Negative Pair Construction |  It may treat the same class of nodes as negative pairs | (1) Nodes with different labels form negative pairs; (2) Labeled and unlabeled nodes (No edges) are mutually treated as negative pairs | \n| Objective | Enhance the representation ability of images | (1) Improve the separability of seen classes; (2) Effectively mine the hidden clustering structures within the graph |\n\nUtilizing the spectral contrastive loss from [29] in ZNC is hindered by data and objectives. [29] targets image-level representations, establishing sample relationships via augmentation. Lacking labels and natural node adjacencies, [29] cannot effectively uncover positive correlations among similar nodes or negative ones between dissimilar nodes.\n\nIt is important to note that our innovations are:\n1. Combine the spectral contrastive learning with the learnable class prototypes to discover the implicit clustering information and realize the semantic migration, thus further alleviating the bias problem.\n2. Make full use of labeling and adjacency to design node spectral contrastive loss mining the implicit clustering information in unlabeled nodes (**Reviewer HC2X also notes this**). \n3. Theoretically, we derive an approximate relationship between the obtained node embeddings and the feature vectors obtained from spectral decomposition.\n\n**> Replay to W2 & Q1: The efficiency and training time**\n\nGiven that your concerns align with Reviewer HC2X, we have responded to this issue in the **General Response**. We sincerely appreciate your time and patience, and kindly encourage you to navigate to that section for our comprehensive response.\n\n**> Replay to W3: The sensitivity of the hyperparameters**\n\nWe have set guidelines for parameter selection that efficiently guide in choosing suitable hyperparameters and facilitate swift model tuning for new data. \n\n**Hyperparameters $\\alpha$ and $\\beta$**\n\nWe analyze the $\\alpha$ and $\\beta$ across three datasets (**Figure 1 in the additional rebuttal PDF**). The $\\alpha$ and $\\beta$ regulate the contribution of labeled and unlabeled samples in overall loss. The results reveal a notable trend: when the value of $\\beta$ is larger than $\\alpha$, the model generally exhibits better performance, with this advantage being particularly prominent when $\\beta = 1$.\n\nThe conclusions meet our expectations.  A larger $\\beta$ can reduce overfitting to seen classes and better exploit the clustering information embedded in unlabeled nodes. Thus, the parameters need to satisfy $\\beta > \\alpha$. This suggests that the $\\alpha$ and $\\beta$ are not sensitive to some extent.\n\n**Hyperparameters $q$** \n\nWhen a sample's pseudo-label probability exceeds the threshold $q$, it updates the class prototypes. The choice of $q$ depends on model and dataset: low $q$ risks noisy pseudo-labels, degrading performance; high $q$ ensures accuracy but limits unlabeled data utilization [1-3].\n\nFortunately, SpeAr achieves satisfactory performance across datasets with $q$=0.5 (Figure 2(c) in Section 4.4). Even on Cora, our result (55.49) at $q$=0.5 significantly surpasses SOTA (48.43), validating the effectiveness of SpeAr. The strategies for handling new datasets are:\n\n1. Set $q$=0.5, assigning lower weights to samples during prototype updates to mitigate noise.\n\n2. Select $q$ using the validation set (**Figure 2 in the additional rebuttal PDF**). Across three datasets, as $q$ varies, the accuracy changes in both the test and validation sets are roughly consistent, confirming the strategy's effectiveness and reliability. This will be discussed in the modified version.\n\n3. In the future, we will focus on advanced threshold selection, including adapting thresholds to class difficulty and dynamically updating them based on model learning.\n\n[1] Fixmatch: Simplifying semi-supervised learning with consistency and confidence\n\n[2] Boosting semi-supervised learning by exploiting all unlabeled data\n\n[3] SemiReward: A General Reward Model for Semi-supervised Learning\n\n**> Replay to W4: Inadequate experimental settings**\n\nWe give all the hyperparameters involved in our SpeAr (**Table 1 in the additional rebuttal PDF**).\n\n**> Replay to Q2: Caution in using seen classes to prevent overfitting** \n\nWe accounted for this in the initial manuscript (Section 4.4) by adjusting the weights of $\\alpha$ and $\\beta$ to further alleviate overfitting issues.\n\nFirst, the seen class information serves as a pivotal semantic bridge during the training phase, facilitating SpeAr's ability to capture and transmit the deep semantic ties between classes, enhancing the model's capacity for generalization towards unseen classes. \n\nSecond, we adjusted the weights of seen and unseen classes within the spectral contrastive loss to mitigate the overfitting to seen classes, setting $\\beta$ to be greater than $\\alpha$. The adjustment also improves the model's adaptability to unseen classes.\n\nRecognizing that our manuscript's lack of detailed explanation may have led to confusion, we will add a detailed discussion in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LS2HueOOlV", "reviewer": "Reviewer_HC2X", "summary": "Zero-shot node classification is a vital task in the field of graph data processing. Prediction bias is one of the primary challenges in zero-shot node classification. This paper employs spectral analysis coupled with learnable class prototypes to discover the implicit cluster structures within the graph, providing a more comprehensive understanding of classes. The authors propose a spectral approach for zero-shot node classification (SpeAr). Establishing an approximate relationship between minimizing the spectral contrastive loss and performing spectral decomposition on the graph, thereby enabling effective node characterization through loss minimization. The class prototypes are iteratively refined based on the learned node representations, initialized with the semantic vectors. Experiments verify the effectiveness of the SpeAr, which can further alleviate the bias problem.", "questions": "1. What are the fundamental differences between traditional zero-shot learning (ZSL) and zero-shot node learning? Furthermore, is there a potential for applying the SpeAr method to zero-shot learning for image classification tasks?\n\n2. In the section 3.5, pre-training is highlighted as a crucial step. Could you elaborate on the necessity of this pre-training phase for the proposed method, and what implications it may have if it were omitted?\n\n3. Could you provide a more detailed explanation of how the proposed approach leads to improved performance, grounded in the theoretical framework outlined in Theory 3.1?\n\n4. The paper mentions refining the prototype update process by selecting the top-s nodes with the highest scores. Could you elaborate on the methodology for determining the optimal value of the hyper-parameter s?", "limitations": "From my opinion, there is no potential negative societal impact of their work.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Overall, I thoroughly enjoyed reading this paper and have a highly favorable impression. \n\n- Learning clustering information for unlabeled nodes represents a pivotal research direction. \n\n- The novel utilization of spectral clustering and the development of a spectral contrast loss function constitute a noteworthy contribution. \n\n- The representation is concise and coherent. The proposed approach demonstrates remarkable efficacy.", "weaknesses": "- High computational complexity. Insufficient experimentation, what it will be if the model is not pre-trained.\n\n- This paper provides a theoretical analysis and there is potential for further elaboration and clarification of the intrinsic correlation between theory and practical performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank the reviewer for carefully reading and for the valuable comments. We greatly appreciate the time and effort you have taken to provide such thoughtful feedback. Below is a point-by-point response to the comments.\n\n**> Response to W1: High computational complexity**\n\nGiven the shared concerns expressed by Reviewer rWPM and yourself, we have formulated a comprehensive and unified response to this comment within our **General Response**. We respectfully request that you consult this General Response for our full explanation and clarification of this issue.\n\n**> Response to W1 and Q2: The important of pre-trained**\n\nThe first phase is our pre-training phase. In the Ablation Study (Section 4.3) of the main body, we validate the model without the first phase (Model2). As shown in the following table, the Model2's results decline across three datasets.\n\n|Dataset|\tModel2|\tSpeAr|\n| - | -|-| \n|Cora|\t52.42|\t60.48|\n|Citeseer\t|48.15|\t59.72|\n|C-M10M|\t50.27|\t54.22|\n\nIn SpeAr, we use the pre-trained model mainly for several reasons:\n\n- Improve the training efficiency of the second phase: In SpeAr, the first phase leverages the unsupervised contrastive loss to train the model. In the second phase, the focus is on training only the final layer of the network. This approach improves training efficiency by reducing GPU memory usage and significantly decreasing the number of gradient updates in the second stage.\n\n- Simulate real-world workflows: In the industry, it is standard practice to first train a model using self-supervised loss on unlabeled data. In the first phase, the model can take advantage of the unlabeled data to learn useful representations from the data and enhance the model's understanding of the data.\n\n**> Response to W2 and Q3: The relationship between theory and practical performance**\n\nGiven that you share the same concerns as Reviewer 7tvx, we have addressed the relationship between theory and performance comprehensively in **General Response**. We kindly request you to refer to the General Response for our detailed response, which we hope will clarify your doubts. Your patience is greatly appreciated.\n\n**> Response to Q1: Differences between traditional zero-shot learning (ZSL) and zero-shot node learning**\n\nWhile both traditional ZSL and zero-shot node classification (ZNC) aim at the recognition of unseen class samples, there are some differences:\n\n1. Data type:\n\n- ZSL: Images, text, or audio data is independent identical distribution. \n\n- ZNC: Graph data, nodes form a network structure through connectivity (edges).\n\n2. Feature representation approaches:\n\n- ZSL: Features are usually extracted by pre-trained feature extractors (e.g. ResNet).\n\n- ZNC: Feature representation not only considers the nodes attributes but also integrates the neighbor information of the nodes and the global graph structure.\n\n3. Semantic migration pathway:\n\n- ZSL: Use semantic information (e.g., word vectors, attribute vectors) to transfer semantic knowledge from seen classes to unseen ones.\n\n- ZNC: Rely on not only semantic information but also relationships between nodes for semantic migration.\n\nTo summarize, traditional ZSL is significantly different from ZNC in terms of data types, feature representations, and semantic migration pathways.\n\nIn future work, applying the SpeAr to ZSL for image classification tasks is feasible. The critical aspect is constructing the neighbor relationships between samples, which can be achieved through feature similarity or image augmentation strategies.\n\n**> Response to Q4: The choice of the hyperparameter $s$**\n\nFor the choice of the hyperparameter $s$, it is set to 1000 across all three datasets. In SpeAr, we found that if $s$ is set too small, its effect on prototype updates is not significant. As shown in the table below, when $s$=100, the results across the three datasets are poor. Based on the experimental results from all three datasets, we found that $s$=1000 achieved better performance consistently.\n\n|$s$\t|Cora\t|Citeseer\t|C-M10M|\n|-|-|-|-|\n|100\t|49.90\t|54.32\t|42.98|\n|200\t|51.10\t|56.93\t|43.66|\n|300\t|55.86\t|57.98\t|45.69|\n|400\t|57.13\t|58.40\t|47.70|\n|500\t|56.67\t|58.87\t|47.93|\n|600\t|55.12\t|59.86\t|49.80|\n|700\t|57.65\t|58.00\t|51.65|\n|800\t|57.15\t|**60.83**\t|53.49|\n|900\t|*58.51*\t|59.47\t|*54.01*|\n|1000\t|**60.48**\t|*59.72*\t|**54.22**|", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UummURH7Vj", "reviewer": "Reviewer_7tvx", "summary": "The manuscript proposes the SpeAr method, which leverages spectral analysis and class prototypes to uncover the implicit clustering structures within graphs, providing a comprehensive understanding of node categories. The proposed method establishes an approximate relationship between spectral contrastive loss and spectral decomposition, optimizing node representations by minimizing the loss and iteratively updating class prototypes. Experimental results demonstrate that SpeAr can effectively mitigate the bias issue.", "questions": "1. In Subsection 3.2, what is the choice of k? Regarding the SpeAr algorithm's ability to distinguish between classes, is this strongly correlated with the value of k? Specifically, is there a tendency for the algorithm's ability to discriminate to increase as the value of k increases? We look forward to deeper understanding and insights into this issue.\n2. The method part of the paper mentions unsupervised spectral contrast loss, but the corresponding formula is not specifically reflected in the paper. Perhaps the formula is more intuitive.\n3. The paper has a very good theoretical analysis. Why spectral contrast loss works for your results.\n\nIn conclusion, zero-shot node classification is a problem well worth studying, and this paper introduces a spectral method designed to alleviate the prediction bias problem of zero-shot node classification. This method has achieved good experimental results for zero-shot node classification. In addition, this paper has relevant theoretical arguments, a novel perspective, and relatively complete theories and experiments.", "limitations": "Similar to questions.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper combines the spectral contrastive learning method with learnable class prototypes to discover implicit cluster information, thereby alleviating the problem of zero-shot node classification prediction bias. It also has relevant theoretical arguments, a novel perspective, and relatively complete theories and experiments.\n2. The two-stage training method in this paper plays a key role in improving the representation quality of the prototype and can further improve the performance of zero-shot node classification.\n3. Employing spectral clustering and the ingenious design of a spectral contrast loss has efficiently addressed the zero-shot node learning challenge, thereby bridging the gap in current research within this field.", "weaknesses": "See the questions listed below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for taking the time to review our manuscript and for the valuable comments. Below is a point-by-point response to the comments.\n\n**> Response to Q1: Discussion on $k$**\n\nIn Section 3.2 of the main body, the eigenvector matrix is obtained by spectral decomposition, $F^{\\ast} = {[q_1, q_2, ..., q_k]}^T \\in R^{N\\times k}$, which serves as a novel representation of the sample. Let $z_i$ be the $i^{th}$ row of the matrix $F^{\\ast}$, It turns out that $z_i$ can serve as desirable node embeddings of $x_i$. Thus, the dimension of the feature space is $k$.\n\nThe classification capability of the SpeAr algorithm is closely related to the embedding dimension $k$. Specifically, the choice of $k$ is influenced by the rank of the adjacency matrix. A larger $k$ allows the algorithm to better capture subtle differences between similar samples. In our experiments with three datasets, we chose $k$=2048 to represent a broad range of variations in the data, not just category distinctions.\n\n**> Response to Q2: The unsupervised spectral contrast loss**\n\nUnsupervised contrastive loss pre-trains the model by leveraging the inherent information within the samples. For more information on the importance of pre-training, please refer to the details in **General Response**. Thank you for your patience.\n\nWhen constructing the unsupervised contrastive loss, each node's positive sample is defined as itself, while the negative samples are all other samples. This approach allows the model to fully utilize the intrinsic information of the data during the pre-training phase, providing more accurate feature representations for subsequent tasks.\n\n**> Response to Q3: The relationship between theory and practical performance**\n\nGiven that your concerns align with Reviewer HC2X, we have provided a comprehensive elaboration on the relationship between theory and performance in General Response. We sincerely invite you to review our response in the **General Response**, which we believe will address your questions. We are deeply grateful for your patience and attentive consideration.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
## **General Response**

We sincerely appreciate the reviewers for their valuable and constructive comments.

We are honored to see that the reviewers recognized the novelty (HC2X, 7tvx), reasonability (rWPM, 7tvx), and significant contributions (HC2X, 7tvx) of our framework. Several reviewers appreciated the clear motivation (rWPM, HC2X, 7tvx) and structure (7tvx) of this paper. Reviewer HC2X acknowledged that our method is simple and effective, outperforming existing methods (rWPM, HC2X, 7tvx) in zero-shot node classification tasks. Reviewer 7tvx commended the theoretical depth of our paper. Additionally, reviewers (rWPM, HC2X) praised our writing.

We have addressed the comments and concerns of each reviewer in our individual responses.

Here are our answers to the common questions:

- Answers to the questions of Reviewer rWPM and Reviewer HC2X about efficiency and computational complexity.

- Answers to the questions of Reviewer HC2X and Reviewer 7tvx about the relationship between theory and performance.

**> Response to Reviewer rWPM and Reviewer HC2X: The efficiency and computational complexity**

Our SpeAr involves computing spectral contrastive loss across all nodes and updating prototypes, which increases computational requirements.

For the large dataset ogbn-arxiv, we adopt the “minibatch + sampling” strategy. Specifically,  the minibatch size is 4096,  and each node samples only a small and fixed number of neighboring nodes, which effectively avoids the risk of memory overflow and also ensures the time efficiency of SpeAr.

In addition, by reducing the number of epochs (1000 $\rightarrow$ 100), the running time of SpeAr is comparable to that of DBiGCN (epoch=10000). However, it is worth noting that the accuracy of SpeAr still significantly outperforms that of DBiGCN. That's because SpeAr mines and utilizes more information. Next, we first give the running time of SpeAr under 1000 epochs for zero-shot node classification (ZNC) in Table 1. Then, we show its running time under 100 epochs in Table 2. Finally, we present the accuracy of SpeAr under 100 epochs in Table 3.

Table 1: Running time of the DGPN, DBiGCN, our SepAr (epoch=1000), First phase, and Second phase (epoch=1000) for ZNC (s means seconds, h means hours).
| | DGPN | DBiGCN | SpeAr (epoch=1000) | First phase | Second phase (epoch=1000) |
| - | -|-| -|-|-|
| Cora | 37s | 138s | 739s | 88s | 651s |
| Citeseer | 90s | 237s | 795s | 103s | 692s |
| C-M10M | 51s | 228s | 854s | 146s |708s |
| ogbn-arxiv |6.2h |26.7h | 86.1h|9.4h |76.7h |

Table 1 compares the runtime of DGPN, DBIGCN, and SpeAr, leveraging the Python implementations from their respective authors. On a unified platform, SpeAr's execution yields higher time consumption. In each epoch, SpeAr requires matrix calculations, loss calculations, and prototype updates.

Remarkably, Table 2 shows that scaling down to 100 epochs significantly reduces runtime for SpeAr, while Table 3 demonstrates that SpeAr maintains accuracy superior to existing methods. This underscores SpeAr's ability to attain commendable performance with fewer epochs due to its comprehensive utilization of information.

Table 2: Running time of the DGPN, DBiGCN, our SepAr (epoch=100), First phase, and Second phase (epoch=100).
| | DGPN | DBiGCN | SpeAr (epoch=100) | First phase | Second phase (epoch=100) |
| - | -|-| -|-|-|
| Cora | 37s | 138s | 169s | 88s | 81s |
| Citeseer | 90s | 237s | 189s | 103s | 86s |
| C-M10M | 51s | 228s | 210s | 146s | 64s |
| ogbn-arxiv | 6.2h|26.7h |20.2h |9.4h | 10.8h|

Table 3: Zero-shot node classification accuracy (%). 
| | DGPN | DBiGCN | SpeAr (epoch=100) |
| - | -|-| -|
| Cora | 33.78 | 45.14 | **55.46** | 
| Citeseer | 38.02 | 40.97 | **52.78** | 
| C-M10M | 41.98 | 45.45 | **50.02** | 
| ogbn-arxiv | 22.37 | 21.40 | **26.23**|

In summary, by adopting the "minibatch + sampling" strategy, we can improve the efficiency of SpeAr on large-scale datasets. We are pleasantly surprised to find that even if the training time is shortened, the SpeAr performance is still competitive. We maintain that, despite an increase in the algorithm's runtime (when epoch=1000), the importance of mining more valuable information significantly outweighs this aspect. We eagerly await your thoughts on this idea and look forward to further discussions on this matter.

**> Response to Reviewer HC2X and Reviewer 7tvx: The relationship between theory and practical performance**

1. Spectral decomposition can mine the intrinsic clustering structure of unlabeled nodes: ZNC aims to identify nodes of classes unseen during the training process. Leveraging the inherent cluster information of unlabeled nodes is essential for enhancing the model's ability to recognize and understand unseen classes. Therefore, we combine spectral analysis with learnable category prototypes to reveal the intrinsic clustering structure in the graph.

2. In SpeAr, the reshaped adjacency matrix $A$ includes label information and node adjacency relationships. SpeAr is actually a method of factorizing the adjacency matrix. 

3. Based on 1 and 2, Theory 3.1 derives the spectral contrastive loss on the graph from the spectral decomposition. The spectral contrastive loss contains several losses, each acting as follows:

   $\mathcal{L}_{1}$ ensure intra-class compactness of labeled nodes.

   $\mathcal{L}_{2}$ mines clustering information of unlabeled nodes using adjacency.

   $\mathcal{L}_{3}$ guarantees interclass separability of labeled nodes.

   $\mathcal{L}_{4}$ constraints the separability of seen and unseen classes.

   $\mathcal{L}_{5}$ treats all remaining unlabeled node pairs (except for node pairs identified as positive nodes in the loss 
   $\mathcal{L}\_{2}$ ) as negative pairs.

Spectral contrastive loss can exploit the structure of the graph data to mine cluster information in unlabeled nodes and ensure the distinguishable between classes in the feature space.

Meta-review: The reviewers collectively agree that the proposed spectral approach for zero-shot node classification (ZNC) is a significant and novel contribution that effectively addresses prediction bias by leveraging spectral clustering and class prototypes. Despite some concerns about computational complexity and sensitivity to hyperparameters, the method's theoretical foundation, clear presentation, and demonstrated effectiveness in experiments are highly appreciated. Based on these strengths, I recommend accepting the paper.
Decision: Accept (poster)