Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Authors: Xunkai Li
Zhengyu Wu
Kaichi Yu
Hongchao Qin
Guang Zeng
Rong-Hua Li
Guoren Wang
Venue: ICML 2025 poster (2025)
Keywords: Data Knowledge Distillation
Graph Neural Network
Directed Graph Learning
Primary area: deep_learning->graph_neural_networks
TL;DR: The first attempt to fully utilize the potential of data to empower directed graph learning through data-centric machine learning.

Abstract: Although directed graphs (digraphs) offer strong modeling capabilities for complex topological systems, existing DiGraph Neural Networks (DiGNNs) struggle to fully capture the concealed rich structural information.
    This data-level limitation results in model-level sub-optimal predictive performance and underscores the necessity of further exploring the potential correlations between the directed edges (topology) and node profiles (features and labels) from a data-centric perspective, thereby empowering model-centric neural networks with stronger encoding capabilities.
    In this paper, we propose **E**ntropy-driven **D**igraph knowl**E**dge distillatio**N** (EDEN), which can serve as a data-centric digraph learning paradigm or a model-agnostic hot-and-plug data-centric Knowledge Distillation (KD) module.
    EDEN implements data-centric machine learning by constructing a coarse-grained Hierarchical Knowledge Tree (HKT) using proposed hierarchical encoding theory, and refining HKT through mutual information analysis of node profiles to guide knowledge distillation during training.
    As a general framework, EDEN naturally extends to undirected graphs and consistently delivers strong performance. 
    Extensive experiments on 14 (di)graph datasets—spanning both homophily and heterophily settings—and across four downstream tasks show that EDEN achieves SOTA results and significantly enhances existing (Di)GNNs.

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Xunkai Li
Zhengyu Wu? Kaichi Yu
Hongchao Qin
Guang Zeng? Rong-Hua Li' Guoren Wang'
Abstract
et al., 2023), graph-level tasks (Zhang et al., 2019; Yang et al., 2022). However, most GNNs are tailored for undi- rected scenarios, resulting in a cascade of negative impacts:
Although directed graphs (digraphs) offer strong modeling capabilities for complex topological systems, existing DiGraph Neural Networks (DiGNNs) struggle to fully capture the concealed rich structural information. This data-level lim- itation results in model-level sub-optimal pre- dictive performance and underscores the neces- sity of further exploring the potential correla- tions between the directed edges (topology) and node profiles (features and labels) from data- centric perspective, thereby empowering model- centric neural networks with stronger encoding capabilities. In this paper, we propose Entropy- driven Digraph knowlEdge distillatioN (EDEN), which can serve as data-centric digraph learn- ing paradigm or a model-agnostic hot-and-plug data-centric Knowledge Distillation (KD) module.
(1) Data-level sub-optimal representation: Due to the com- plex structural patterns present in the real world, the absence of directed topology limits the captured relational informa- tion, thereby resulting in sub-optimal data representations with inevitable information loss (Koke & Cremers, 2024; Geisler et al., 2023; Mackawa et al., 2023); (2) Model-level inefficient learning: The optimization dilemma arises when powerful GNNs are applied to sub-optimal data. For in- stance, undirected GNNs struggle to analyze the connective rules among nodes the entanglement homophily and heterophily (i.e., whether connected nodes have similar fea- tures or same labels) (Luan et al., 2022: Zheng et al., 2022; Platonov al., 2023) due to neglect the valuable directed topology (Rossi et al., 2023; Maekawa et al., 2023; Sun et al., 2024). This oversight compels the undirected methods to rely heavily on well-designed models or tricky theoretical assumptions to remedy the neglect of directed topology.
EDEN implements data-centric machine Iearn- ing by constructing a coarse-grained -rrained Hierarchical Knowledge Tree (HKT) using the proposed hierar- chical encoding theory, and refining HKT through mutual information analysis node profiles to guide knowledge distillation during training. As general framework, EDEN naturally extends to undirected graphs and consistently delivers strong performance. Extensive experiments on 14 (di)graph -paunnstt--pannnnn both homophily and heterophily settings-ann and across four downstream tasks show that EDEN achieves SOTA results and significantly enhances existing (Di)GNNs.
To break these long-standing limitations, Directed GNNs (DiGNNs) are proposed to extract implicit data knowledge from directed graphs (Rossi et al., 2023; Sun et al., 2024; Li et al., 2024a) by exploring directionality of edges as criti- cal topological patterns in guiding more efficient message passing of DiGNNs and enhancing representation learning. Despite ad advancements in existing model design consider- ing asymmetric topology, these methods still fail to fully explore the potential correlations between directed topol- ogy and node profiles the data level. More importantly, this potential correlation extends beyond directed edges and high-order neighbors to unseen but pivotal structural pat- terns (referred to as digraph data knowledge).
1. Introduction
Therefore, we emphasize revealing this data knowledge from data-centric perspective to improve the learning util- ity of model-centric approaches fundamentally. Specifically, (1) Topology: unlike undirected graphs, directed topology offers node pairs groups more diverse range of connec- tion patterns, implying abundant structural knowledge; (2) Profile: digraph nodes present greater potential for more sophisticated profile knowledge caused by directed edges when compared to nodes in the undirected graph that often present with predominant homophily (Ma al., 2021; Luan
Recently, Graph Neural Networks (GNNs) have achieved SOTA performance across node- (Wu et al., 2019; Hu et al., 2021; Li et al., 2024b), link- (Zhang & Chen, 2018; Tan
'Department of Computer Science and Technology. Bei- jing Institute of Technology, Beijing, China *Alibaba Group, HangZhou, Zhejiang, China. Correspondence to: Rong-Hur Hong-Huall Li <lironghuabit@@ 126.com>.
Proceedings of the 4 International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
al., 2022: Zheng et al., 2022). The two above perspectives form the basis of digraphs (i.e., the entanglement of directed topology and node profiles). The core of our approach is dis- entangling this complexity with our proposed data-centric hierarchical encoding system. For the motivation and key insights behind this framework, please refer to Sec. 2.2.
13 Directed Stracture
Towwtedge Refremert [terstiv w Me Prowtedge Gorarstion (his -ftremt
ID besedd Supertsed Leaming ef and Estiattion Any DiRAN
Entropy Meosrement
BB
Topslop- bud Merk
Grendy Algenthe for IKT
Knoeledge
Topolay
Frow, Trarsfer lorer
HET ked sereet
HKT Layerniux
G = (V.E)
Excelesee
Hienorchical Encoding System Loyer
To this end, we propose Entropy-driven Digraph know EEdee distillatioN (EDEN). As a general Knowledge Distillation (KD) strategy, EDEN seamlessly integrates data knowledge into model training as supervised information to obtain the optimal embeddings for downstream tasks. Specifically, EDEN first employs directed structural measurement as a quantification metric to capture the natural evolution of di- rected topology, thereby constructing a Hierarchical Knowl- edge Tree (HKT) (topology perspective). Subsequently, EDEN refines the HKT with fine-grained adjustments based on the Mutual Information (MI) of node profiles, regulating the knowledge flow (profile perspective). Based on this, EDEN can be viewed as a new data-centric DiGNN or a hot-and-plug data-centric online KD strategy for existing DiGNNs. Notably, while we highlight the importance of EDEN in extracting intricate digraph data knowledge, it can naturally extend to undirected scenarios and exhibit satisfac- tory performance. More details can be found in Sec. 4.1.
Figure I. The overview of our proposed hicrarchical encoding the- ory with HKT layer-wise KD. Its trainable core involves digraph learning function within every HKT layer and MI neural estima- tion across layers (illustrated using leaf nodes and their parents), Different colors and dotted lines represent distinct labels.
Node-level Classification. Suppose Vi2 is the labeled set, the semi-supervised node classification paradigm aims to predict the labels for nodes in the unlabeled set Vn with the supervision of Vj. For convenience, we call Node-C.
Link-level Prediction. (1) Existence: predict if (u, v) & exists in the edge sets; (2) Direction: predict the edge direction of pairs of nodes u, e for which either (u, v) E &
or (u, u) E E; (3) Three-class link classification (Link-C): classify (u, v) E E (v, u) E E, or (u, v), (v, u) &
2.2. Hierarchical Encoding Theory in Structured Data
Inspired by the information theory of structured data (Li & Pan, 2016), let G be a real-world digraph influenced by natural noise. We define its information entropy H from topology and profile perspectives, where H. determines the true structure T, and data knowledge IC is concealed in T. The assumptions about these definitions are as follows:
New Perspective. To the best of our knowledge, EDEN is the first attempt to achieve hierarchi- cal data-level KD. It offers a new and feasible perspective for data-centric graph ML. (2) Unified Framework. EDEN facilitates data-centric digraph learning through the estab- lishment of fine-graines HKT from topology and profile perspectives. It contributes to discovering unseen but valu- able structural patterns concealed in the digraph to improve data utility. (3) Flexible Method, EDEN can be regarded as a new data-centric digraph learning paradigm. Furthermore, can also serve as model-agnostic hot-and-plug data- centric KD module, seamlessly integrating with existing DiGNNs to improve predictions. (4) SOTA Performance. Extensive experiments demonstrate that EDEN consistently outperforms the best baselines (up to 3. 12% higher). More- over, it provides a substantial positive impact on prevalent (Di)GNNs (up to 4.96% improvement).
Our contributions. (I)
Assumption 2.1.
The information entropy H is captured by the directed topology and profile-based hierarchical encod- ing system, reflecting the uncertainty of complex systems.
Assumption 2.2. mizing H. reflecting the natural organization of nodes.
The true structure T is obtained by mini-
Assumption 2.3. The data knowledge IC forms the foun- dation of G and is concealed in the T, which is used to optimize the trainable hierarchical encoding system.
Based these assumptions, we adhere to the traditional hierarchical encoding theory (Bymee & Russon, 1998; Dit- tenbach et al., 2002; Clauset et al., 2008) to establish a novel data-centric digraph learning paradigm shown in Fig. 1. This paradigm standardizes the evolution of structured data in physical systems, inspiring the notion of decoding this natu- rally structured knowledge for analyzing complex digraphs. In other words, this trainable encoding system progressively captures the information needed to determine nodes, such as their positions uniquely. From this, the encoded result constitutes knowledge XC residing within the true structure T. Subsequently, applying KD on extracted IC from T opti- mizes the encoding system to achieve iterative training. The above concepts form the core of our motivation.
2. Preliminaries
2.1. Notations and Problem Formulation
We consider a digraph 9 = (V,E) with [V] = n nodes, [ = m edges. Each node has feature vector of size f and a one-hot label of size c, the feature and label matrix are represented as X E / and Y E Raxe G can be described by an asymmetrical adjacency matrix A(u, v). D diag (di,--.. dn) is the corresponding degree matrix. Typical digraph- based downstream tasks are follows.
2

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Notably, the directed structural measurement and node MI in Fig. I aim uncover the topology and profile complexity. Based on this, we efficiently compress information, reduce redundancy, and reveal hierarchical structures that capture subtle data knowledge often overlooked by previous studies. In other words, we minimize uncertainty and noise in G, re- vealing the underlying true structure T, which captures the layered organization of the data's inherent evolution. This T allows us effectively decode the underlying knowledge K, corresponding to the HKT in EDEN. This theoretical hypothesis has been widely applied in recent years, driving significant research advancements in graph contrastive learn- ing (Wu et al., 2023; Wang et al., 2023) and graph structure learning (Zou et al., 2023; Duan et al., 2024).
curring patterns. To effectively estimate MI, MINE (Belg- hazi al., 2018) adopts the DV (Pinsky, 1985) represen- tation to approximate the KL divergence, which is closely related to MI. It performs neural MI estimation by parame- terizing the function space with a neural network and opti- mizing a tight lower bound via gradient descent. Building on these insights, DGI (Veličković et al., 2019) introduces graph Infomax to guide contrastive learning. GMI (Peng et al., 2020) further enhances feature aggregation by maxi- mizing the MI between a node and its local neighborhood. CoGSL (Liu et al., 2022) leverages MI to optimize graph view generation and fusion to guide graph structure learning.
3. Methodology
In this paper, we adopt a data-centric perspective, which we believe has been overlooked in previous studies. Specifi- cally, we investigate the potential of data-level KD as su- pervised information to enhance model-centric (Di)GNNs. The core intuition behind our approach is that data quality often limits the upper bound for model performance (Guo et al., 2025; Zheng et al., 2023; Liu et al., 2023). By lever- aging HKT, we can break this data-level limitation. This is particularly relevant for digraphs, where intricate directed causal relationships demand deeper exploration. However, our approach can also be naturally extended to undirected graphs. For further discussion on our proposed HKT and hicrarchical graph clustering, please refer to Appendix A.I.
The core idea of EDEN is to fully leverage the digraph data knowledge to empower model training. As data-centric online KD framework, EDEN achieves mutual evolution between teachers and students (i.e., parent and child nodes in the HKT) shown in Fig. 2. To avoid confusion between the data-level online KD and the model-level offline KD (i.e., large teacher model and lightweight student model), we provide detailed explanation in Appendix A.2.
Step I: Knowledge Discovery: (a) To begin with, we employ directed topology measurement as quantification metric to construct a coarse -oarneee HKT; (b) Based on this, we per- form node MI neural estimation. Through gradient descent, we regulate the knowledge flow to obtain fine-grained HKT.
2.3. Digraph Representation Learning
Step 2: Knowledge Distillation: Then, we denote parent and child nodes within the same corrected partition teachers and students to achieve online KD. Specifically, we propose node-adaptive knowledge generation and transfer operators.
To obtain digraph node embeddings, both spectral (Zhang et al., 2021c; Lin & Gao, 2023; Koke & Cremers, 2024; Li et al., 2024a) and spatial (Tong et al., 2020b;a; Zhou et al., 2022; Rossi et al., 2023; Sun et al., 2024) methods are proposed. Specifically, implement spectral convolution on digraphs with theoretical guarantees, the core is to de- pend on holomorphic (Duong & Robinson, 1996) filters or obtain a symmetric (conjugated) digraph Laplacian based on PageRank (Andersen et al., 2006) or magnetic Lapla- cian (Chung, 2005). Regarding spatial methods, researchers draw inspiration from the ppessage-passing mechanisms that account for directed edges. They commonly employ inde- pendently learnable weights for in- and out-neighbors to fuse node representations (He et al., 2022b; Kollias et al., 2022: Sun et al., 2024).
Finally, we generate leaf predic- tions (i.e.. original digraph nodes) for downstream tasks. In this process, to harness rich knowledge from the HKT, we employ random walk to capture multi-level representations from their parents and siblings to improve predictions.
Step 3: Leaf Prediction:
3.1. Multi perspeppeseeettvee Knowledge Discovery
In the context of digraph learning, original data have two pivotal components: (1) Topology describes the intricate connection patterns among nodes; (2) Profile uniquely iden- tifies each node. If knowledge discovery focuses on only one aspect, it would lead to coarse-grained knowledge and sub-optimal distillation. To avoid this, EDEN first conducts topology mining to enrich subsequent profile mining, and collectively, establish a robust foundation for effective KD.
2.4. driven MI Neural Estimation
Information entropy was introduced to quantifyy uncertainty in communication systems (Shannon, 1948). Motivated by it, MI measures the dependence between two random vari- ables. Based on this, Infomax (Linsker, 1988) maximizes the MI between inputs (features) and outputs (predictions), concentrating the encoding system more on frequently oc-
Topology-awand structural measurement. In a highly con- nected digraph, nodes frequently interact with their neigh- bors. By employing random walks (Pearson, 1905), we can capture these interactions and introduce entropy as a
3

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Topology Measurement
Step Knowledge Discovery (Leaf Node in 2D Plane for Illustration)
Profile MI Neural Estimation
(V,8)
Structural Entropy Metric
Intro- partition Node-wise Similarity
Gready Algorithm
Inter partition Node-wise Similarity
(a) Coarse grained Topology Mining
(b) Fine-grained Profile Mining
Knowledge Generation
Step& 2 Knowledge Distillation (Blue for Illustration)
Step Leaf Prediction
Generation Operator
O Transfer Operator
Knowledge Transfer
Parent
Sibling
Child
General (Free)
Link (Sibling)
MT
Node (Porent)
Hierarchical Knowledge Tree
Knowledge Generation
Knowledge Transfer
HKT-based Random Walk
Figure 2. The overview of our proposed EDEN.
measure of topology uncertainty (Li & Pan, 2016). The topology uncertainty is substantiated in real-world obser- vation that relational data is often polluted with structural noise, such as spurious edges and missing links, and our ob- jective is to quantify it by Shannon entropy and minimize it to reorganize node affiliations, thereby effectively denoising the graph. Specifically, we can quantify oieedimensional structural measurement of G by leveraging the stationary distribution of its degrees d and the Shannon entropy, which is formally defined as:
where t is the parent of and is the root node of the HKT, g/"h and of are the number of directed edges from other partitions to the current partition and from the current partition to other partitions, at the node level.
Coarse-grained HKT construction. In contrast to the topology measurements defined in previous work (Li & Pan, 2016), EDEN addresses the limitations of forward-only random walks by incorporating reverse probability. This modification is motivated by the non-strongly connected nature of most digraphs, where the proportion of complete walk paths declines sharply after only five steps (shown in Appendix A.4). This decline suggests that strictly adhering to edge directions in walks (forward-only) fails to capture sufficient information beyond the immediate neighborhood of the starting node. Furthermore, we add self-loops for sink nodes to prevent the scenario where the adjacency matrix might be a zero power and ensure that the sum of landing probabilities is 1. Based on this, we utilize Eq. (3) as met- ric and employ a greedy algorithm (De & Temlyakov, 1996) to seek the optimal HKT that minimizes uncertainty. For a detailed algorithm, please refer to Appendix A.5.
Jun
jout
dout
H H')
log
log
(1)
777
m
where dm and dout are in and out-degrees of digraph node. Based on this, to achieve high-order topology mining, let P = (&,, X2 Xe} be a partition of V. where X' de- notes a community. To this point, we can define the two- dimensional structural measurement of by P as follows:
HG) min (L)()) HlurlGi}. (9.
vol (V2) m
/unt
log
du jun/out v)2)
91 log
vol (V,) T7
Profile-aware node measurement. As previously pointed out, node proñies play an equally pivotal role in digraph learning, which means that the topology measurement alone is insufficient to reflect the true strueture. Therefore, we aim to leverage node profiles to fine-tune HKT for KD. The key insight is to emphasize high node similarity within the same partition while ensuring differences across distinct partitions. This is retain authority in the parent nodes (teachers) and avoid the reception of misleading knowledge by the child nodes (students). To achieve our targets, we introduce intra- and inter- mnrrtpartttion node MI neural estimation. The former (3) retains nodes with higher MI within the current partition. These nodes not only serve as effective representations of
vol (V3)
m
where vol(V) Evey dm /d V/ and 9j the nodes and the number of directed edges with end-point/start-poin in the partition j. Notably, real-world digraphs commonly exhibit hierarchical structure, extending Eq. (2) to higher dimensions. Consequently, we leverage h-hhiight partition tree T (Appendix A.3) to obtain h-dimensional formulation:
(G) min (T-HHeight(TT=
{ITT + N.un(19)
g. in/out vol(V)
log vol (t) vol ((+))
HTT oun (G) =- Σ
vIET,0A

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
the current partition but also inherit partition criteria based on topology measurement. The latter identifies nodes in other partitions that effectively represent their own partitions while exhibiting high MI with the current partition. We can adjust the affiliations of these nodes to improve HKT.
where (.) is the activation function. Since solving I across the entire function space F is practically infeasible, we employ a neural network Fu)(;-) parameterized by
Theorem 3.3. Through the optimization of w, we obtain C(Q)== 7(0) as the GAN-based node MI neural estimation for every partition within fine-grainet HKT:
Partition-based MI neural estimation. Before introducing our method, we provide a formalized definition as follows. For current partition Xp. we first sample a subset Qp consist- ing of Kp nodes from X, and other partitions Xg at the same HKT height (more details can be found in Appendix A.6). Then, we employ a criterion function C-)) to quantify the in- formation of S2p, aiming to find the most informative subset for generating knowledge about X. by solving the problem ca ((2,)), subject to [Spp] = Kp- In our implemen- tation, we formulate C(2p)) for Xr, based on the neural MI estimator between nodes and their generalized neighbor- hoods, capturing the neighborhood representation capability of nodes. Based on this, we derive the following theorems related to MI neural estimation for structured data, guiding the design of a criterion function for HKT partitions.
C(Q)= = 7(9) =
=70)
Σ BEE
logo (Fur (Xu: XNT))
max W0
[9]
(6)
log (1 ( (Fu [X,:XN]]
+ max w
[2
(u.B)ER
The two terms capture the dependency and difference be- Iween selected nodes and their neighborhoods.
Fine-grained HKT correction. Based on the above theo- rems, we instantiate the intra-partition MI:
pintro == Qintra (Wi (M (), (M (XNT)))) (7)
Theorem 3.1. Let be the HKT in digraph G (V,E). For any selected node v E X2 and u E Xg in the subset Syp we define their generalized neighborhoods
where Qintra is an embedding function designed to quantify node MI by maximizing intra intra-aariiiion X₂ similarity, M is model-agnostic digraph learning function, and Wi and W2 are embedding functions for selected nodes and their generalized neighborhoods. Building upon this, we extend Eq. (7) to the inter-partition scenario, enabling the discovery of potential nodes u that exhibit high MI with Xp and inherit the directed structure measurement criteria of Xg:
NºT Xx and No. = Xp J Xg. Given v and No. as an example, consider random variables fo and fNT as their unique node (sets) features, the lower bound of MI between v and its generalized neighborhoods is given by the KL di- vergence between the joint distribution P (for fNT) (ff=fNT) = P (Jo =xv INTJ = =XNT) and the product of marginal distributions Pj= @ &PINT can be defined as follows:
Finter:- == Qinter (X.M) We (M(X)T))) (8) W
( (Jor = DKI. (P (Jun IPPs, @ @PINT)
Notably, the above equations share W1 and W2, as they are both used for encoding the current node and corresponding generalized neighborhoods. In our implementation, Q and W are instantiated as MLP and the linear layer. Furthermore, we combine it with Sec. 3.2 to reduce complexity. Detailed proofs of the theorems can found Appendix A.6-A.8.
2 sup FEF
E P F (Xer XNT)N
(4)
(xxx
sup FEF
Ex Rx x Xxx NT
where 8 represents the randomly selected node in Qpp except for v. This lower bound is derived from the f-divergence representation based on KL divergence. F is an arbitrary function that maps a pair of the node and its generalized neighborhoods to a real value, reflecting the dependency.
3.2. Node-adaptive Knowledge Distillation
Knowledge Generation. After considering the distinctness of nodes, we obtain 2pp for the current partition X, by solv- ing Eq. (6), where Qp comprises K nodes selected from X, and other partitions Xg. Now, we compute affinity score S for each sampling node in Stp based on their unique roles vx given by HKT, where i1 is the nodes from the cur- rent partition, and 22 is the nodes obtained by performing partition- by- partitiin sampling of the other partitions. The sampling process is limited by the number of nodes in Xp.
Theorem 3.2. The lower bound in Theorem 3.1 can be converted to f-divergence representations based on non-KI. divergence. This GAN-like divergence for structured data is formally defined as:
DKL (P(f. FNT) 3PIT) ~ IIANN (FTT
2 sup FEF
E BP((nndNT) [logo (F (x,,X)])]
S., = -g.=o(2n ( )), W3(M(XNT) ))), (5)
Soy, = (Qinter (M(X..)...) W2(M(X NNT) ))), (9)
EP log (1g(1- (F((Xx
+ sup FEF
S = @iitre (WI W2(M(X (xxx)))))
5

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
where Sa and Sena are used to discover the knowledge related to the current partition. However, this strategy often causes over-fitting. Therefore, we introduce S to bring diverse knowledge from other partitions. Specifically, we aim to identify and emphasize nodes that, while represent- ing other partitions, exhibit significant differences from the current partition by S.g = max(3 (S02.1 1..1 S Finally, we obtain the parent representation of X, by Xp Sn Xa,-
next node r. The transition probability is set as follows:
1 Prws parent
1/srus sibling
Prw
1 l/ew child
0, otherwise
(11)
Then, we concat the k-step random walk results (i.e., node sequence) to obtain Ph. for each leaf node. After that, the leaf-centered prediction and overall optimization with flexible KD and MLP instantiated Q.e are formally defined as (please refer to Appendix A.9 for complexity analysis);
Knowledge Transfer. In this section, introduce person alized knowledge transfer from the parent node Xp (teacher) to the child nodes X. (student) under partition Xp. The key insights are as follows: (1) For parent nodes, not all knowl- edge is clearly expressible, implying that class knowledge hidden in embeddings or soft labels may be ambiguous. (2) For child nodes, each node has unique digraph context, causing various knowledge requirements. Therefore, we consider the trade-off between the knowledge held by the parent node and the specific requirements of child nodes.
(x,Y) + aLkat
C = Ceross-entropy
Y Node (v) Softmax (Orade (p) (12) Y Link (u, v) SSotmaa Qlink (Pim-ull I u))
Specifically, we first refine the knowledge hidden in the parent node through Qporent- Then, we aim to capture the diverse requirements of child nodes in knowledge transfer by Qotai to achieve personalized transfer. Similar Sec. 3.1, employ MLP to instantiate Q. To this point, we have built an end-to-end framework for the mutual evolution of
3.4. Lightweight EDEN Implementation
As a data-centric framework, EDEN implements HKT- driven data KD. This framework offers new insights and tools for advancing data-centric graph ML. However, scala- bility remains bottleneck in our approach, and we aim to propose feasible solutions to enhance its efficiency. Specifi- cally, we implement a lightweight EDEN outlined below.
teacher and student by the data-level online KD loss:
U =0 (emmreen (-Ex
xxx log (xxx: (xx²))
(10)
Lightweight Coarse-grained HKT Construction. As de- tailed in Algorithm 1-2 of Appendix A.5, we introduce Monte Carlo methods, which select potential node options rather than optimal ones before detaching and merging. This approach involves running multiple Monte Carlo simula- tions, where nodes are randomly chosen in each run to generate various candidate solutions, avoiding exhaustive enumeration of all branches for subsequent greedy selec- tion. An optimal or near-optimal solution is then selected for execution without rendering expensive computational cost, offering an alternative to scalable demands.
Lxd = || (u - Qersia (XFr, Il..
3.3. Random walk-based Leaf Prediction
Now, we have obtained representations for all nodes in the HKT, the focus shifts to generating leaf-centered predictions for various downstream tasks. To improve performance, a natural approach is to leverage the multi-level represen- tations, including siblings and higher-level parents of the current leaf node, to provide more informative context. Accordingly, we employ the tree-based random walk to ob- tain this embedding sequence. However, within a given receptive field, the number of possible paths often exceeds the number of nodes, making it computationally impracti- cal to consider all paths- particulaall as the receptive field expands. To address this, and to capture more informative and task-relevant patterns with fewer paths, we define walk rules that are tailored the specific requirements of the downstream task. Specifically, we concentrate on sampling siblings (8yw) to capture same-level representation for link- level tasks. Conversely, for node-level tasks, we prioritize sampling from parents (Prw) or children (Crw) to acquire multi-level representations. Consider a random walk on edge Ct,s: currently positioned node s and moving to the
Lightweight Fine-grained HKT Construction. For node MI neural estimation, computational efficiency can be fur- ther optimized using incremental training and prototype rep- resentation for label-specific child and parent nodes. This training and embedding representation method will signifi- cantly reduce the computational overhead.
Lightweight Layer-wise Digraph Learning Function. We can obtain node representations through weight-free feature propagation, computationally efficient embedding method that has proven effective in recent studies (Wu et al., 2019; Zhang et al., 2022; Li et al., 2024b). Through this design, we significantly reduce the number of learnable parameters and achieve efficient gradient updates.
6

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Table 1. Model performance (%) in three directed link-level downstream tasks.
Datasets (-)
Slashdot
WikiTalk
Tasks (-+)
Exislence
Direction
Link-C'
Existence
Direction
Link-C
Models ())
AUC
AP
AUC
AP
ACC
AUC
AP
AUC
AP
ACC
GCN
88. 18.4
68.620.1
10.1=0.1
90.240.1
83.840.2
92.440.1
92.3+0.0
86.5+0.2
87.1
84.620.2
GAT
88.1+0.2
88.4+0.1
90.4+0.2
90.5+0.1
83,5+0.3
0OM
OOM
OOM
00M
OOM
NSTE
90.6+t011 1
90.8+0.0
92.2=0.1
92.4+0.0
85,4+0.2
94,4#0.1
94.620.1
90.7#0.1
90.0+0.0
90.4#0.1
DirGGNN
90.420.1
90.5+0.0
92.0=0.1
91.810.1
85,240.2
94.720.2
94,320.1
90.940.1
90.3
90.610.2
Nat
90.340.1
90,240.1
92.2+0.2
92.410.1
85.340.1
0OM
OOM
QOM
00M
0OM
MGC
90.140.11
90.440.0
92.1=0.1
92.340.1
85.040.1
94.5#0.1
94.2s0.0
90.6+0.1
90.240.0
90.1
EDEN
91,8+0.1
92.0=0.0
93,3+0.1
93.1+0.00
87.1+0.2
95,4+0.1
95.8+0.1
91,5+0,0
91.7+0.1
91,0+0.11
Table 2. Link-C ACC and others AUC (%) three directed link-level downstream tasks.
Datasets
Tasks
GCNII
GATv2
AGT
DGCN
DIMPA
D-HYPR
DIGCN
MagNet
HoloNet
EDEN
Existence
84.01+0.22 84,58=0.33
84.50=0.24
87.65+0.20
88. 1..06.0.22
87, 87.99=0.24 87.65+0.28 88. 88.05+0.21 87 97.80=0.22
90.84+0.19
CoraMI.
Direction
83.25#0.36
84.94=0.00
85.57=0.51
90.43+0.49
90.88+0.50
90. 90.94+0.54 89.75#0.71
90.83+0.49
89.83=0.57
92.36220488
Link-C
70.43+0.55
71.24=0.58
71.23=0.47
72.55+0.48 72.86+0.55
72.9140.38
72.53+0.56
72.96+0.42
72.74=0.56
18.1540.54
Existence
76.2410.46 76.86#0.35
76.7240.38
79.65540.9
79.6540.38
79. 79.8440.29
79.3240.33
79.80+0.24
79.32+0.30
82.24+0.37
CiteSeer
Direction
72.95 8.2 74.08=0.56 73.7640.65
88. 1240.73
88.42.0.70 70
88.7540.63
88.19 19+0.88 88.67 08.6670.45
88. 78.77440.. /48
90.56++0.40
Link-C
62.3740.88
63.21
62.53=0.57
64.0240.56
64.21+0.43
64.30+0.37
63 (3.9240.59
64.03+0.40
63 63.9440.36
66.73+0.57
Existence
92.31=0.10
92.46=0.1
92.22+0008
92.441+0.15
93.78=0.15
93.75+0.14
93.42+0.12 93. 93.6240.10
93 8440440.00
94.93+0.10
Tolokers
Direction
18.1410.16
88.27=0.10 10 89.12=0.07
88. 12.9220.12
89. 89.9010.11
89. 39.9440.10 10 89.6820.09 89.8320.09
89. 89.7620.11 11
91.52=0012
Link-C
78.10#0.11
78.29=0.19 19 78.72=0.15
79.74+0.08
80. 00.84
80.79*0.09
80.52+0.10
80.78+0.8
80.51=0.12
82.67=0.13
Existence
63.37=0.72
63.78=0.90 90 63.92=0.74
65.67:0.40
66. 06.2840.32 3.2 66.31+0.35
66.39=0.43
66.27=0.34
65 86-666046
68.81+0.41
Empire
Direction
49.56:1..0 50.6440.78 76 50.38:0.70
53. 53.26:00.2
53. 33.9240.36
53.8740.42
53.91+0.50
53.8440.39
53. 13.7990.45
55.60+0048
Link-C
53.41+0.90
13+0844
53.43=0..99
58.0540.38
58.5640.49
58.6440.48
58.6 58.6440.54
58.56 58.5640.26
58 58.33+0.35
60.74+0.39
Existence
74.68+0.54
74.83#0.64
75.08=0.33
76. 76.6440.24
76. 8440.22 77.3940.32
77.30+0.29
77.31
77. 77.1240.26 022
79.52+0.27
Rating
Direction
79.3240.41
79.65=0.4p
.2 79.5640.37
$2.3440.33
82.91
83.5810.29
83.62+0.33
83. 83.5620.27
83 83.30 0.33
85.1990.29
Link-C
59.95+0.62
60.27=0.58
59.37=0.40
63, 03.223
63.78+0.30
64.3340.36
64.28#0 40 64.32+0.30
64.32+0.32
66.37=0.35
Existence
83.1420.23
82.54=0.31
82.21=0.18
84.40+1.11
85. 85.1910.21
85.1340.23
85.02+0.31
85.29=0.19
85 85.25=0.20
87.24+0.23
Arxiv
Direction
89.20=0.27
89. 89.13=0.29
89.47=0.28
93.05+0.16
93.41=0.19
93.2440.20
93. 18:0.25
93.3720.14
93.40=0.15
94.48=00116
Link-C
75.97+0.21 75.60=0.18 75.29=0.15
78. 08.24
78. 78.90+0.20 78.7440.14 78.69=0.26 78.97=0.23
78.93=0.21
80.16=0.21
Table 3. Test accuracy (%) in directed Node-C.
4.1. Performance Comparison
Models
ComMI.
CiteSeer
WikiCS
Tolokers
Empire
Rating
Arxiv
GCNII
80.840.5
62.5400.
48.3
78.5.0.1
76.3+0.4
42.3+0.5
65.450.3
A New Digraph Learning Paradigm. To answer Q1, we report the performance of EDEN, a deta-centric DiGNN, in Table 1, Table 2, and Table 3. We evaluate EDEN on three link-level downstream tasks designed to assess its ability to predict both the existence and directionality of edges. As shown in Table 1 and Table 2, EDEN consistently achieves SOTA performance across diverse datasets, highlighting its strength in extracting structural and node-level semantics from directed graphs. The node-level classification results in Table 3 further demonstrate EDEN's effectiveness on both homophilous and heterophilous datasets, indicating its ability disentangle directed topological structure from nodes profiles. Compared baselines that intermittently reach second-best performance, EDEN achieves improve- ments of 2.78% and 2.24% on node-level and link- tasks, respectively. Notably, details of the HKT layer-wise digraph learning funetion-servingg both as a novel directed graph learning paradigm and a plug-and-pla, module for existing DiGNNs-ard are provided in Appendix A.
GATv2
81.310.
628409
78.0404
78.840.2
78.2s01.
43.820.6
66.720.3
AGT
81,24088
62.9008
78.3403
78.540.2
77.6.0.7
43.6:0.4
66.240.4
DOCN
82.240.5
63.540.7
78.420.3
78.740.3
78.7
44.750.6
66.920.2
DIMPA
82/410.6
64.02088
78.810.4
78.920.2
79.0200.
44.620.5
67.120.3
D.HYPR
82,7604
63.840.7
78.7402
79.2+0.2
78.8+0.5
44.9+0.5
66.8+0.3
DIGON
82000.6
63,94055
79.040:
79.1
78.4+0.6
44.3+0.7
67.1
MagNet
82.210.5
6422006
78.9402
79.020.2
78.8a11.4
44.750.6
67.320.
HolaNei
82.5+0.5
1407
79.240.
79.4+0.2
78.7+0.5
44.5+0.6
67.5+0.2
EDEN
84610.5
65.840.6
81,420.3
81.340.2
$1.150.6
46.3=0.4
69.720.3
4. Experiments
In this section, we offer a comprehensive evaluation to ad- dress the following questions: Q1: How does EDEN per- form as a new data-centric DIGNN? Q2: As a hot-and-plug data online KD module, what is its impact on the prevalent (Di)GNNN? Q3: If EDEN is effective, what contributes to its performance? Q4: What is the running efficiency of EDEN? Q5: How robust is EDEN when dealing with hy- perparameters and sparse scenarios? We introduce datasets, baselines, and experiment settings in Appendix A. 1...-A.13.
7

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Table 4. Node-C test accuracy (%) gains brought by EDEN in Di(GNNs) under Di(graphs).
Models
CoraML
CiteSeer
WikiCS
Arxiv
Photo
Computer
PPI
Flickr
Improv.
OptBG
81.5=0.7
62.4=0.7
77.9=0.4
66.4+0.4
91.5=0.5
82.8+0.5
57.2=0.2
50.9+0.3
12.75%
OpIBG+EDEN
82.8=0.6
64.620.8
79.4=0.3
67.9+0.4
93.9=0.6
84.920.6
59,8=0.3
52.8+0.4
NAG
81.2=0.9
62.5=0.9
78,3=0.3
65.9+0.5
91.3=0.7
83,1
57.1=0.2
51.2=0.4
12.54%
NAG+EDEN
83.0+0.9
64.8+0.7
79.8+0.4
67.3+0.4
93.6+0.8
85.2+0.5
59.2+0.2
52.5+0.4
DIMPA
82.4=0.6
64.0=0.8
78.
67.1
91.4#0.6
82.4+0.5
56.7+0.3
50.5+0.3
1..32%
DIMPA+EDEN
85,4+0.5
66.9+0.7
82.240.5
69.9
1
85.5
59.5+0.4
52.940.2
Dir-GNN
82.6#0.6
64,5=0.6
19.1
66.9=0.4
91.1=0.5
82.9#0.6
56,8=0.3
50.8+0.4
1.4.8%
Dis-GNN+EDEN
85,9=0.4
67.2=0.5
82.8+0.3
70,5+0.3
93.8=0.5
84.8+0.7
09,4
53.1
HoloNet
82.5+0.5
100.7
79.240.3
67.5+0.2
90.8+0.5
83.0+0.6
57.0+0.3
51.0=0.4
114466
HoloNec+EDEN
86.0+0.4
67.5#0.6
82.6+0.2
70.8+0.3
93.7+0.5
85.3+0.5
59.5+0.5
53,440.5
by solely focusing on the current partition, achieving higher accuracy and lower variance. Module (2) affirms our key insight in Sec. 3.2, improving HKTTbbaee KD.. Module (3) indirectly underscores the validity of the EDEN, as the multi-level representations embedded in the HKT provide
Table 5. Ablation study performance (%)
Datssetst=-))
Tolokers (ACC)
Slashdox (AUC)
Modules(L)
Node C
Link
Existence
Direction
EDEN
81.33200.2
82.0720.1
91.82+011
93.2920.1
wlo Diverse Knowledge
80,90150.4
82.222003
91.1020.2
92, 9690.2
who Personalized Transfer
50,8450.2
82.3650.1
91. 1950.1
93.015011
wis lodge Divtiliation Loss
Then Random Walk
80.6740.3
80,01+0.3
82.1840.2
81.10-0.1
91.1610.1
9084+0.1
92.7740.1
92.25+0.1
beneficial information for various downstream tasks. Finally,
Rating (ACC)
Epinions (AU/C
module (4) unifies the above modules into an end-to-end optimization framework to empower digraph learning.
Modules(1)
Node-C
Link-C
Existesce
Disection
EDEN
46.3320.4
66.37s0.4
93.48:0.1
89.4010.1
w/o Diverse Knowledge
45.7640.6
66.0340.5
93.11+0
39 02.02+0.1
who Persontaliedd Transder
45.92+0.3
65.94+0.3
93.08+0.1
80 00.04+0.1
wio Tree-lined Racdoon Walk
45.84150.4
65.72e11.5
93.02:0.1
88. 99:0.1
4.3. Efficiency Comparison
Knowledge Distillaion Loss
45.4520.5
65.0Mei).
92.5720.1
38.612011
To answer Q4, we present the running efficiency report in
Fig. 3, where EDEN is primarily divided into two segments: (1) The pre-processing step depicted in Fig. 3(a) showcases coarse-graince HKT construction, with the x-axis represent- ing predefined tree height h; (2) The end-to-end training step depicted in Fig. 3(b). The x-axis denotes the selection of tree height h and sampling coefficient K introduced by Sec. 3.1 and Sec. 3.2. Since the pre- precesscessnngg is independent of model training, the computational bottleneck introduced by the coarse-grained HKT construction is alleviated, re- ducing constraints on deployment scalability. Additionally, the lightweight implementation in pre-pr pre-processing further mitigates it. Meanwhile, benefiting from the lightweight fine- fine-grained HKT construction and personalized layer-wis. digraph learning function, EDEN exhibits a significant ad- vantage in training costs compared to existing baselines shown in Fig. 3(b)-c)) Due to space constraints, additional details regarding the model convergence efficiency during the training process can be found in Appendix A.14.
A Hot-and-phug Online KD Module. Subsequently, to answer Q2, we present performance gains achieved by in- corporating EDEN as a hot-and-plug module into existing (Di)GNNs in Table 4 (deployment details can be found in Appendix 11.. Based on the results, we observe that EDEN performs better on digraphs and DiGNNs compared to undirected ones. This is because more abundant data knowledge is inherent in digraphs, providing stronger coding potential for DiGNNs. EDEN is designed to meet this specific demand, thus showcasing superior performance. Notably, the performance of EDEN as a hot-and-plug mod- ule exceeds its performance as a self-reliant method in some cases. This is attributed to the adoption of lightweight EDEN for running efficiency. While this approach sacrifices some accuracy, it significantly enhances scalability.
4.2. Ablation Study
To answer Q3, we present ablation study results in Table 5, evaluating the effectiveness of: (1) Diverse knowledge in
4.4. Robustness Analysis
Eq. (9) for over-fitting; (2) Node-adaptive personalized transfer for KD (Eq. (10)); (3) Tree-based random walk for leaf prediction (Eq. (11)); (4) KD loss function for the gra- dient interaction between teachers and students (Eq. (10)).
Hyperparameter Selection. To answer Q5, we first an- alyze the impact of hyperparameter selection on running efficiency and predictive performance based on Fig. 3(a) and (b). Our observations include: (1) Higher HKT height h leads to substantial increase in the time complexity for greedy algorithm during pre-process; (2) Larger sam- pling coefficients & indicate additional computational costs
Experimental results demonstrate a significant improvement by combining these modules, validating their effectiveness. Specifically, module (1) mitigates over-fitting issues caused
8

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Gpilic
MagNei Dokdher
Srrppeece
AGT
DGGU
Table 6. Comparison in Test accuracy (%) and Running Time (See)
Rumning 5 3
Lightwight
Lighewight Acceierte
Bascime Aut
in Node-C between Original EDEN and Lightweight EDEN.
Aveage
48%
Anc
Aec
51
79.0
Node-C
Tolokers(ACC) Toloker<(Tim)) Rating (ACC) Rating (Time)
Ligatneade
385+
031
12
Bernce 20
EDEN (Ori.)
82, 1+0.3
240.8+6.5
4,.820.3
132.344.9
EDEN (Light.)
81.3+0.2
12.1
46.3=0.4
57.6+2.2
10
3-2
10-1. 1042
DIGNOGNN Busfises
(5) Pre Pre-process for Step
(b)
Training
(c) Training Efficiency
Figure 3. Efficiency of Node-C on Empire.
Lightweight version of EDEN serves promising alter- native in more scalability -emanddng scenarios. To validate this, we compare nove-level classification results between the full and Lightweight versions in Table 6. While the Lightweight variant shows a slight drop in performance, it significantly reduces computational overhead, saving run- time by 101.5% on Tolokers and 18.6% on Rating datasets. The performance gap stems from the Monte Carlo method's approximationn-bssed search, though the Lightweight ver- sion remains computationally efficient.
85
Tert Accurey
80
NC
75
bam
78
Dinm
70
miu
76
meN
Potnt of Renwinng Nude Feraes
Paoeet Rousinng Siges
Denert Rerwinsg per Clrs
Feature Sparsity
Edge Sparsity
(e) Label Sparsity
Figure 4. Node-C performance on CoraML.
due considering more nodes in the knowledge genera- tion, especially pronounced with increased height h; (3) Ap- propriately increasing h and k for fine-grained distillation significantly improves performance. However, excessive in- crease leads to apparent optimization bottlenecks, resulting in sub-optimal performance. In addition, we further dis- cuss the implementation details of HKT-based random walk for leaf prediction and KD loss factor in Appendix 1.14. This involves investigating the impact of transition proba- bilities between distinct identity nodes (i.e., parent, sibling, and child) during the sequence acquisition on predictive performance and further analyzing the effectiveness of KD.
5. Conclusions, Limitations, and Future Work
In this paper, we propose a general data-centric (di)graph online KD framework, EDEN. It achieves fine-grained data knowledge exploration abiding with the hierarchical encod- ing system proposed in Sec. 2.2. Comprehensive evaluations demonstrate significant all-around advantages. We believe that implementing data-centric graph KD through the tree structure is a promising direction, as the hierarchical struc- ture effectively captures the natural evolution of graphs. However, it must be acknowledged that the current EDEN framework has significant algorithmic complexity, including multi-step computations. Despite the lightweight implementation, scalability challenges persist when applied to billion-level graphs. Therefore, our future work aims to simplify the hierarchical data-centric KD theory and de- velop a user-friendly computational paradigm. For instance, we intend to explore the possibility of integrating the HKT construction with graph partitioning techniques to enable parallel processing, and facilitate its practical deployment in broader industrial applications.
Sparsity Challenges. Subsequently, we provide sparse ex- perimental results in Fig. 4. For stimulating feature sparsity, we assume that the feature of unlabeled nodes is partially missing. In this case, methods that rely on the quantity of node representations like D-HYPR and NAG are severely compromised. Conversely, DiGCN and MGC exhibit ro- bustness, as high-order feature propagation partially com- pensates for missing features. As for edge sparsity, since all baselines rely on high-quality topology to empower their model-centric neural architectures, their predictive perfor- mance is not optimistic. However, we observe that EDEN exhibits leading performance through fine-graing fine-grained digraph data-centric knowledge mining. To stimulate label sparsity, we change the number of labeled samples for each class and acquire results that follow a similar trend to the feature- sparsity scenarios. Building upon these observations, EDEN comprehensively improves both the predictive performance and robustness of the various baselines.
A Acknowledgement
This work is supported by the National Natural Sci- ence Foundation of China (NSFC) No.U2241211 and U24A20255. Rong-Hua Li is the corresponding author of this paper.
Impact Statement
Evaluation Lightweight EDEN Navigating the com- plexity of digraphs as a data-centric learning framework, EDEN demonstrates strong predictive performance and scalability, as evidenced by its evaluation on million-scale datasets such as Arxiv and Wikitalk (see Table 1 and Ta- ble 3). As discussed in See. 3.4, the Monte Caseo
This paper presents work whose goal is to advance the field of Machine Learning. There are few potential societal con- sequences of our work, none which we feel must be specifi- cally highlighted here.
9

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
References
Duan, L., Chen, X., Liu, W., Liu, D., Yue, K., and Li, A. Structural entropy based graph structure learning for node classification. In Proceedings of the Association for the Advancement of Artificial Intelligence, AAAI, 2024.
Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. Optuna: A next- next-generation hyperparameter optimization framework. In Proceedings of the ACM SIGKDD Confer- ence on Knowledge Discovery and Data Mining, KDD, 2019.
Duong, X. T. and Robinson, D. W. Semigroup kernels, poisson bounds, and holomorphic functional calculus. Journal of Functional Analysis, 142(():89-128) 1996.
Andersen, R., Chung, F., and Lang, K. Local graph partition- ing using pagerank vectors. Anmual IEEE Symposium on Foundations of Computer Science, FOCS. IEEE, 2006.
Feng, K., Li, C., Yuan, Y., and Wang. G. Freekd: Free- direction knowledge distillation for graph neural net- works. In Proceedings of the ACM SIGKDD Conference Knowledge Discovery and Data Mining, KDD, 2022.
Belghazi, M. I., Baratin, A., Rajeswar, S., Ozair, S., Bengio, Y., Courville, A., and Hjelm, R. Mine: mutual infor- mation neural estimation. In International Conference on Machine Learning, ICML, 2018.
Geisler, S., Li, Y., Mankowitz, D. J., Cemgil, A. T., Günnemann, S., and Paduraru, C. Transformers meet directed graphs. In International Conference on Machine Learning, ICML. PMLR, 2023.
Bojchevski, A. and Günnemann, S. Deep gaussian em- bedding of graphs: Unsupervised inductive learning via ranking. In ICLR Workshop on Representation Learning Graphs and Manifolds, 2018.
Goodfellow, I., Pouget- J., Mirza, M., Xu, B., Farre-Farley. D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets, Advances in Neural Informa- tion Processing Systems, NeurIPS, 2014.
Brody, S., Alon, U., and Yahav, E. How attentive are graph attention networks? International Conference Learn- ing Representations, ICLR, 2022.
Grave, E., Bojanowski, P., Gupta, P., Joulin, A., and Mikolov, Learning word vectors for 157 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). Euro- pean Language Resources Association (ELRA), 2018.
Byrne, R. W. and Russon, A. E. Learning by imitation: A hierarchical approach. Behavioral and Brain Sciences, 21 (5):667-684, 1998.
Chen, J., Gao, K., Li, G., and He, K. Nagphormer: A tokenized graph transformer for node classification in large graphs. In International Conference on Learning Representations, ICLR, 2023.
Guo, Y. and Wei, Z. Graph neural networks with learnable and optimal polynomial bases. 2023.
Guo, Y., Bo, D., Yang, C., Lu, Z., Zhang, Z., Liu, J., Peng, Y., and Shi, C. Data-centric graph learning: survey, IEEE Transactions on Big Data, 11(1):1-20, 2025. doi: 10. D0.1100/BDDAAA2224444888
Chen, M., Wei, Z., Huang, Z., Ding, B., and Li, Y. Simple and deep graph convolutional networks. In International Conference on Machine Learning, ICML, 2020.
Chen, Y., Bian, Y., Xiao, X., Rong, Y., Xu, T., and Huang, On self- distillnnn graph neural network. Proceedings of the International Joint Conference on Artificial Intelli- gence, IJCAI, 2021.
He, Y., Perlmutter, M., Reinert, G., and Cucuringu, M. Ms- gnn: A spectral graph neural network based on a novel magnetic signed laplacian. In Learning on Graphs Con- ference, LoG, 2022a.
Chung, Laplacians and the cheeger incquality for directed graphs. Annals of Combinatorics, 9:1-19, 2005.
He, Y., Reinert, G., and Cucuringu, M. Digrac: Digraph clustering based on flow imbalance. In Learning on Graphs Conference, LoG, 2022b.
Clauset, A., Moore, C., and Newman, M. E. Hicrarchical structure and the prediction of missing links in networks. Nature, 453(7191):98-100 2008.
He, Y., Zhang, X., Huang, J., Rozemberczki, B., Cucuringu, M., and Reinert, G. Pytorch geometric signed directed: A software package on graph neural networks for signed and directed graphs. Learning on Graphs Conference, LoG. PMLR, 2023.
DeVore, R. A. and Temlyakov, N. Some remarks on greedy algorithms. Advances Computational Mathe- matics, 5:173-187, 1996.
Hiriart-Urruty J.-B. and Lemaréchal, C. Fundamentals of convex analysis. Springer Science & Business Media, 2004.
Dittenbach, M., Rauber, A., and Merkl, Uncovering hier- archical structure in data using the growing hierarchical self-organizing map. Neurocomputing, 48(1-4):199 216, 2002.
10

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovec, J. Open graph benchmark: Datasets for machine learning on graphs. Advances in Neural Information Processing Systems, NewrlPS, 2020.
Likhobaba, D., Pavlichenko, N., and Ustalov, D. Toloker Graph: Interaction of Crowd Annotators. 2023. doi: 10.5281//eeodd0.7227755 URL https://githan com//toloka/TolokerGzz
Hu, Y., L., Wang, Y., Wu, Y., Zhao, Y., Yan, C., Yin, J., and Gao, Y. Adaptive hypergraph auto-encoder for rela- tional data clustering- IEEE Trunsuctions on Knowledge and Data Engineering. 2021.
Lin, L. and Gao, J. A magnetic framelet-based convolutional neural network for directed graphs. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP, 2023.
Kipf, T. N. and Welling, M. Semi-supervised classifica- tion with graph convolutional networks. In International Conference on Learning Representations, ICLR, 2017.
Linsker, R. Self-organization in perceptual network. Com- puter, 21(3):105-117 1988.
Liu, G., Inae, E., Zhao, T., Xu, J., Luo, T., and Jiang. M. Data-centric learning from unlabeled graphs with diffu- sion model. Advances in Neural Information Processing Systems, NeurIPS, 2023.
Klicpera, J., Bojchevski, A., and Günnemann, S. Predict then propagate: Graph neural networks meet personal- ized pagerank. In International Conference on Learning Representations. ICLR, 2019.
Liu, N., Wang, X., Wu, L.,, Chen, Y., Guo, X., and Shi, C. Compact graph structure learning via mutual information compression. In Proceedings of the ACM Web Conference, WWW, 2022.
Koke, C. and Cremers, D. Holonets: Spectral convo- lutions do extend to directed graphs. In The Twelfth International Conference on Learning Representations, 2024. URL htwpt:tt////...... htttt:::///peerreviiiee net/ forum? id=EhmEwfavokt
Luan, S., Hua, C., Lu, Q. Zhu, J., Zhao M., Zhang, S., Chang, X.-W,, and Precup, D. Revisiting heterophily for graph neural networks. Advances in Neural Information Processing Systems, NeurIPS, 2022.
Kollias, G., Kalantzis, V., Idé, T., Lozano, A., and Abe,
N. Directed graph auto-encoders. In Proceedings of the Association for the Advancement of Artificial Intelligence,
AAAI, 2022.
Ma, X., Chen, Q. Wu, Y., Song, G., Wang, L.,, and Zheng, B. Rethinking structural encodings: Adaptive graph trans- former for node classification task. In Proceedings of the ACM Web Conference, WWW. 2023.
Leskovec, J. and Krevl, A. Snap datasets: Stanford large network dataset collection. 2014.
Leskovec, J., Huttenlocher, D., and Kleinberg, J. Signed networks in social media. In Proceedings of the SIGCHI conference on human factors in computing systems, 2010.
Ma. Y., Liu, X., Shah, N., and Tang. J. Is homophily a neces- sity for graph neural networks? International Conference on Learning Representations, ICLR, 2021.
Lhoest, Q. Villanova del Moral, A., Jernite, Y., Thakur, A., von Platen, P., Patil, S., Chaumond, J., Drame, M., Plu, J., and Tunstall. Datasets: A community library for natural language processing. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing: System Demonstrations, pp. 175-184. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021 111166331/022leeem/dddeeee222
Maekawa, S., Sasaki, Y., and Onizuka, M. Why using either aggregated features or adjacency lists in directed or undi- rected graph? empirical study and simple classification method. arXiv preprint arXiv:2306. 08274, 2023.
Massa, P. and Avesani, P. Controversial users demand local trust metrics: An experimental study on epinions. com community. In Proceedings of the Association for the Advancement of Artificial Intelligence, AAAI, 2005.
Li, A. and Pan, Y. Structural information and dynamical complexity of networks. IEEE Transactions on Informa- tion Theory, 62(6):3290-3333 3339,033333000000 2016.
Mernyei, P. and Cangea, C. Wiki-cs: A wikipedia-basee
benchmark for graph neural networks. arXiv preprint arXiv:2007.02901, 2020.
Li, X., Liao, M., Wu, Z., Su, D., Zhang, Li, R.-H., and Wang, G. Lightdic: A simple yet effective approach for large-scale digraph representation learning. Proceedings of the VLDB Endowment, 2024a.
Nowozin, S., Cseke, B., and Tomioka, R. f-gan: Training generative neural samplers using variational divergence minimization. Advances in Neural Information Process- ing Systems, NeurlPS, 2016.
Li, X., Ma, J., Wu, Z., Su, D., Zhang, W., Li, R.-H., and Wang, G. Rethinking node-wise propagation for large- scale graph learning. In Proceedings of the ACM Web Conference, wwW, 2024b.

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Ordozgoiti, B., Matakos, A., and Gionis, A. Finding large balanced subgraphs in signed networks. In Proceedings of the ACM Web Conference, WwW, 2020.
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. Graph attention networks. In International Conference on Learning Representations, ICLR, 2018.
Pearson, K. The problem of the random walk. Nature, 72 (1865):294 294, 1905.
Veličković, P., Fedus, W., Hamilton, W. L., Liò, P., Bengio, Y., and Hjelm, R. D. Deep Graph Infomax. In Interna- tional Conference Learning Representations, ICLR, 2019.
Peng. Z.,, Huang, W., Luo, M., Zheng, Q., Rong, Y., Xu, T., and Huang. J. Graph Representation Learning via Graph- ical Mutual Information Maximization. In Proceedings of Web Conference, WWW, 2020.
Wang. K., Shen, Z., Huang, C., Wu, C.-H., Dong, Y., and Kanakia, A. Microsoft academic graph: When experts are not enough. Quantitative Science Studies, 1(1):396-413, 2020.
Pennington, J., Socher, R., and Manning, D. Glove: Global vectors for word representation. In Proceedings of Conference on Empirical Methods in Natural Language Processing. EMNLP, 2014.
Wang. Y., Wang, Y., Zhang, Z., Yang, S., Zhao, K., and Liu, J. User: Unsupervised structural entropy- -based robust graph neural network. In Proceedings of the Associa- tion for the Advancement of Artificial Intelligence, AAAI, 2023.
Pinsky, R. On evaluating the Monsker-varadhan i-function. The Annals of Probability, pp. 342-362, 1985.
Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., and Prokhorenkova, L. A critical look at the evaluation of gnns under heterophily: are we really making progress? International Conference on Learning Representations, ICLR, 2023.
Wu, F., Souza, A., Zhang, T. Fifty, C., Yu, T., and Wein- berger, K. Simplifying graph convolutional networks. In International Conference on Machine Learning, ICML., 2019.
Rossi, E., Charpentier, B., Giovanni, F. D., Frasca, F., Günnemann, S., and Bronstein, M. Edge directionality improves learning on heterophilic graphs. in Proceed- ings of The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD Workshop, 2023.
Wu, J., Chen, X., Shi, B., Li. S., and Xu, K. Sega: Structural entropy guided anchor view for graph contrastive learning. In International Conference Machine Learning, ICML, 2023.
Yang, M., Shen, Y., Li, R., Qi, H., Zhang, Q. and Yin, B. A new perspective the effects of spectrum in graph neural networks. In International Conference on Machine Learning, ICML, 2022.
Shannon, C. E. mathematical theory of communication. The Bell System Technical Journal, 27(3):379-423, 1948.
Shchur, O., Mumme, M., Bojchevski, A., and Günnemann, S. Pitfalls of graph neural network evaluation. Relational Representation Learning Workshop, NeurlPS 2018, 2018
Zeng, H., Zhou, H., Srivastava, A., Kannan, R., and
Prasanna, V. Graphsaint: Graph sampling based inductive learning method. In International conference learning representations, ICLR, 2020.
Sun, H., Li, X., Wu, Z., Su, D., Li, R.-H., and Wang, G. Breaking the entanglement of homophily and heterophily in seme-super node classification. In International Conference Data Engineering, ICDE, 2024.
Zhang, H., Lin, S., Liu, W., Zhou, P., Tang, J., Liang, X.. and Xing, E. P. Iterative graph self-distillation. IEEE Transactions Knowledge and Data Engineering, 2023.
Tan, Q. Zhang, X., Liu, N., Zha, D., Li, L.,, Chen, R., Choi, S.-H., and Hu, X. Bring your own view: Graph neural networks for link prediction with personalized subgraph selection. In Proceedings of the ACM International Con- ference on Web Search and Data Mining, WSDM, 2023.
Zhang, J., Hui, B., Harn, P.W.., Sun, M.-T., and Ku, W.-. Mge: A complex-valued graph convolutional network for directed graphs. arXiv e-prints, pp. arXiv-2110, 2021a.
Zhang, M. and Chen, Y. Link prediction based on graph neu- ral networks. Advances Neural Information Processing Systems, NeurIPS, 2018.
Tong, Z., Liang, Y., Sun, C., Li, X., Rosenblum, D.. and Lim, A. Digraph inception convolutional networks. Advances in Neural Information Processing Systems, NeurIPS, 2020a.
Zhang, W., Miao, X., Shao, Y., Jiang, J., Chen, L., Ruas, O., and Cui, B. Reliable data distillation on graph convolu- tional network. In Proceedings s of the ACM on Manage- ment of Data, SIGMOD, 2020.
Tong, Z., Liang, Y., Sun, C., Rosenblum, D. S., and Lim, A. Directed graph convolutional network. arXiv preprint arXiv:2004. 13970, 2020b.
12

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Zhang, W., Jiang, Y., Li, Y., Sheng, Z., Shen, Y., Miao, X., Wang, L.., Yang, Z.,, and Cui, B. Rod: reception-aware online distillation for sparse graphs. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD,, 2021b.
Zheng, X., Liu, Y., Bao, Z., Fang, M., Hu, X., Liew, A. W.-C., and Pan, S. Towards data-centric graph chine learning: Review and outlook. arXiv preprint 1xXiv:2309109977 2023.
Zhou, H., Chegu, A., Sohn, S. S., Z., De Melo, G., and Kapadia, M. D-hypr: Harnessing neighborhood modeling and asymmetry preservation for digraph representation learning. Proceedings of the ACM International Confer- ence on Information and Knowledge Management, CIKM, 2022.
Zhang, W., Sheng, Z., Yang, M., Li, Y., Shen, Y., Yang. Z., and Cui, B. Nafs: A simple yet tough-to-beat base- line for graph representation learning. In International Conference on Machine Learning, ICML, 2022.
Zhang, X., He, Y., Brugnone, N., Perlmutter, M., and Hirn, M. Magnet: A neural network for directed graphs. Advances in Neural Information Processing Systems, NeurIPS, 2021c.
Zhu, Y., Li, J.. Chen, L., and Zheng, Z. The devil is in the data: Learning fair graph neural networks via partial knowledge distillation. In Proceedings of the ACM In- ternational Conference on Web Search and Data Mining, WSDM, 2024.
Zhang, Z., Bu, J., Ester, M., Zhang, J., Yao, C., Yu, Z., and Wang, C. Hierarchical graph pooling with structure learning. arXiv preprint arXiv: 10///99/1000 1.05954, 2019.
Zou, D., Peng, H., Huang, X., Yang, R., Li, J., Wu, J., Liu, C., and Yu, P. S. Se-gsl: A general and effective graph structure learning framework through structural entropy optimization. In Proceedings of the ACM Web Conference, WwW, 2023.
Zheng, X., Liu, Y., Pan, S., Zhang, M.. Jin, D.. and Yu. S. Graph neural networks for graphs with heterophily: A survey. arXiv preprint arXiv: 200.02220. 2022.
13

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
A. Outline
The appendix is organized as follows:
A.1 HKT Construction and Hierarchical Graph Clustering.
A.2 Data-level Online Knowledge Distillation EDEN.
A.3 The Definition of partition tree.
A.4 Breaking the Limitations of Single-direction Random Walks.
A.5 Greedy Algorithms for Partition Tree Construction.
A.6 The Proof of Theorem 3.1.
A.7 The Proof of Theorem 3.2.
A.8 The Proof of Theorem 3.3.
A.9 Algorithm Complexity Analysis.
A.10 Dataset Description.
A.11 Compared Baselines.
A.12 Hyperparameter Settings.
A.13 Experiment Environment.
A.14 Extend Experimental Results.
A.1. HKT Construction and Hierarchical Graph Clustering
Although the HKT construction process may superficially resemble hierarchical clustering, it is crucial to emphasize that HKT is fundamentally distinct. Unlike traditional clustering methods, HKT leverages topology-driven structural entropy- dynamic metric grounded in the information theory of structured data-to to extract deeper structural insights from the graph. This methodology transcends the limitations of static clustering techniques, offering a more nuanced and context-aware understanding the underlying graph structure. Moreover, EDEN incorporates profile-oriented node through neural estimation pivotal criterion for HKT construction. This integration enables more fine- mne-grained and profile-aware analysis of node relationships, uncovering intricate patterns that static methods may overlook. As result, the multi-gra uulti-gramularity quantification criteria established by our approach not only deviate significantly from conventional hierarchical clustering but also introduce novel and innovative perspective for understanding complex graph data (see Sec. 3.1 for more details).
While traditional hierarchical clustering can reveal the layered structure of a network, it is not directly applicable to the complexities of (di)graph learning. EDEN, on the other hand, utilizes HKT as a foundational framework to enable the development of Iearnable knowledge generation and transfer mechanisms that can be seamlessly integrated with existing (Di)GNN architectures. This integration provides a novel way enhance model learning by effectively capturing and utilizing the hierarchical structure of directed graphs. Furthermore, we have designed random walk-based leaf prediction mechanism, tailored to various graph-based downstream tasks, ensuring that our approach is robust and adaptable to different application scenarios (for more technical details, refer to Sec. 3.2-3.3).
A.2. Data-level Online Knowledge Distillation in EDEN
Graph KD typically follows model-level, offline macher-studenn framework. In this setup, knowledge is transferred from large, pre-trained teacher GNN to a more compact and efficient student model, such as a smaller GNN or MLP. The teacher captures complex patterns and representations within the graph. The student, rather than learning directly from ground truth labels, learns from the teacher's ' soft predictions or intermediate representations. This approach allows the student model to replicate the teacher's performance while significantly reducing computational complexity.
14

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
With the rapid advancement of KD, it has expanded into multiple model-level KD variants. These include self-distillation, where single model simultaneously acts as both the teacher and student, enhancing its own learning process (Chen et al., 2021; Zhang et al., 2023), and online distillation, where both teacher and student models are continuously updated throughout the training process (Zhang et al., 2021b; Feng et al., 2022). These innovations reflect the growing diversity in how knowledge transfer can be applied beyond the initial teacher-student (large model to lightweight model) framework.
In this paper, we focus specifically on data-centric graph KD, which emphasizes uncovering the latent knowledge embedded in graph structures, using data samples as the medium for distillation (Zhang et al., 2020: Zhu al., 2024). In the EDEN framework, parent and child nodes within the HKT assume the roles of teacher and student, respectively. This enables knowledge transfer through their representations in hierarchical manner. Our approach aligns with the principles of data-level online KD, leveraging the topological relationships between nodes to drive more effective distillation.
A.3. The Definition of partition tree
To define high-dimensional measurements of directed structural information, we introduce a partition tree T of digraphs, which can also be regarded as the coarse-grained HKT without profile-oriented refinement (i.e., knowledge discovery (a) from a topology perspective only). Notably, community detection or clustering can be understood as a hierarchical structure, specifically a 3-layer partition tree. In this structure, the leaf nodes represent the individual nodes from the original graph, while their parent nodes serve as virtual nodes that represent entire communities. To make it easier to understand, we first give an example of two-dimensional directed structural measurement of the graph, (G), where we consider a digraph G G= (V,8) and its 2-order partition P (, . of node sets V. Building upon this, we interpret P through a 2-height partition tree T as follows.
To begin with, we introduce the root node À and define a set of nodes Tx V/ as a subset of the root node À in the 2-height partition tree T. Notably, in this two-dimensional directed structural measurement, the nodes in the 2-height partition tree have only three types of identity information:
(1) the root node (h- = 0), which does not exist in the original digraph but is used to describe the partition tree; (2) the successor nodes (h- 1), which are not present in the original digraph but are employed to characterize leaf nodes;
(3) the leaf nodes (h = 2). which represent the original digraph nodes.
= 1,.... C. Naturally, we can extend the concept associated with the root to successor nodes Φi, which are directly related to the coarse partitioning of leaf nodes X, Thus, we define To, = Xg Now, for each o,, we introduce |2,,] immediate successors denoted (x()) for all j E (1, (1...., [21)), and each successor db (j) is associated with an element in X,. Thus, we define To, (f) as the singleton of node in To. = Xz
Then, we introduce C immediate successors for the root denote Φ = X(i), =
To this point, T is partition tree of height 2, and all its leaves are associated with singletons. For any node a E T, Ta is the union of T3 for all & values (immediate successors) of a, and the union of T, for all nodes with values at the same level of the partition tree T constitutes a partition of V. Hence, the partition tree of digraph is a set of nodes, each associated with a nonempty subset of nodes in digraph 9, and can be defined as follows:
Definition A.1. (partition tree of Digraphs): Let G = (V,E) be a connected digraph. We define the h-height partition tree T of G with the following properties:
(1) For the root node X, we define the set T3 V as the collection of nodes with heights less than X.
(2) For each node a E T, the immediate successors of a are denoted as a(j) for j ranging from I to a natural number N, ordered from left to right as j increases.
(3) For any natural number S and each non-leaf node a * X, the set (Ta [(())) = i} forms a partition of V, where
h((a) denotes the height of (note that the height of the root node is 0).
(4) For each leaf node in T, Ta is singleton, indicating that Tu contains a single node from V.
(5) For any two nodes a, B E T at different heights, we use a C 3 or 3 C a to denote their hierarchical relationship.
(6) For a C 8 or B C a, we employ- - and + to further describe this hierarchical relationship within the same partition.
Specifically, if a C 3 with h(a)= = h(3) + 1, then 3 represents the child nodes of 3. Conversely, if 3 c a with h(3) = h((x)) +1 1, then B+ denotes the parent node of B3. (note that for every nea-leaf node a + X, (a -- M(a = h 11)
15

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
100
100
Coraml
Coraml
Proportion (%)
CiteseerDir
CiteseerDir
80
80
Arxivdir
Arxivdir
Amazon-Ratt Rating
Proportion
Amazon Raazonn-Rating
60
60
40
40
20
20
0
0
0
10
15
20
10
15
20
Walk Lengths
Walk Lengths
(a) Walk with Circles
(b) Walk without Circles
Figure 5. The visualization experiments of the interruption issuc of single-direction random walk digraphs. The circle represents special topology where node can walk back to itself, and its existence alleviate walk interruption. The y-axis denotes the proportion of non-walk interruptions.
(7) For each Ta is the union of Ti for all 3 such that 3+ a. Thus, Ta = Ua++=a?3 Ts.
According to Definition A. for given digraph G, we compute the h-dimensional directed structural information mea- surement 7" (9) G by Eq. (3) while simultaneously identifying a h-height boarse-grained HKT T. The above process adheres to the following principles:
(1) The h-dimensional structural information measurement (g) of a digraph 9 is achieved or approximated through the h-dimensional hierarchical partition tree T of G;
(2) H² (G) serves the guiding principle for the formation of the h-ddmensional coarse-grained HKT T by minimizing the uncertainty or non-determinism inherent in the h-dimensional structures of G;
(3) T, functioning as coarse- graiind HKT for G, encompasses the rules, regulations, and orders governing g. This HKT derived by minimizing the random variations present h-dimensional structures the digraphs, with these variations being determined by our h-dimensional directed structural information measurement.
Based on the above principles, the h-dimensional structural measurement of digraphs, provided by the h-height partition tree, serves as metric enabling to comprehensively or maximally identify the h-dimennsional structure while mitigating the impact of random variations in the digraphs. Mcanwhile, (G) excellently facilitates the complete extraction of order from unordered digraphs, allowing us to discern order from disorder within structured data. Remarkably, our definition retains all properties of the digraphs, providing robust support for the thorough analysis of structured data.
A.4. Breaking the Limitations Single-direction Random Walks
Utilizing simple random walks (SRW) on digraphs introduces unique challenges due to the inherent structure of these graphs. common issue arises when the random walk encounters nodes with no outgoing edges, causing the walk to terminate prematurely. To better understand and visualize this limitation, we apply SRW starting from each node across four different digraphs. As the walk length increases, we track the proportion of complete paths relative the total sequences, as shown in Fig. 5(a). To further assess the impact of graph cycles, design modified SRW that excludes cycles and conduct the same experiment, with results presented in Fig. 5(b).
This investigation highlights a key limitation of random walks on digraphs: strictly following edge directions leads to frequent interruptions the walk. Due to the non-strongly connected nature of most digraphs, the proportion complete walks drops sharply after just five steps. This indicates that random walks on digraphs typically fail to gather information beyond the immediate neighhorhood of the starting node, limiting their ability to capture long-range dependencies. Moreover,
16

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
when we eliminate the influence of cycles, the proportion of uninterrupted sequences declines even further, underscoring the difficulty of maintaining continuous paths in digraphs and further highlighting the limitations of SRWs (forward-only) in exploring deeper graph structures. It is evident that this significantly hinders the ability of structural entropy to capture topologiaal uncertainty, reducing the effectiveness of H. (g) and leading to sub-optimal coarse- grerseegraindd HKT.
A.5. Greedy Algorithms for Partition Tree Construction
The primary impetus for developing the greedy partition tree construction algorithm lies in the quest for an effective method to construct hierarchical tree structures from digraph data while simultaneously minimizing the complexity and uncertainty associated with the underlying relationships. In complex systems represented by digraphs, directed structural entropy serves as a key metric to gauge the disorder and intricacy within the network. By harnessing the concept of directed edge structural entropy minimization, the algorithm aims to derive hierarchical trees that capture essential structural characteristies while promoting simplicity and interpretability. In a nutshell, the design principles of our proposed algorithm are as follows
(1) Directed edge structural entropy definition: entropy within the context of the digraph mentioned in Sec. 3.1. This metric quantifies the uncertainty and disorder associated with the relationships between nodes in the digraph.
The algorithm hinges on rigorous definition of directed edge structural
(2) Greedy selection strategy: At its core, the algorithm employs a greedy strategy, iteratively selecting directed edges that contribute most significantly to the reduction of directed structural entropy. This strategy ensures that each step in the tree construction process maximally minimizes the overall disorder in the evolving hicrarchy.
(3) Hierarchical tree construction: The selected directed edges are systematically incorporated into the growing tree structure, establishing hierarchical order that reflects the inherent organization within the graph. This process continues iterations until a coherent and informative tree representation is achieved.
(4) Complexity considerations: The algorithm balances the trade-off between capturing essential structural information and maintaining simplicity. By prioritizing directed edges that significantly impact entropy reduction, it aims to construct trees that are both insightful and comprehensible.
In conclusion, the greedy partition tree construction algorithm for digraph data, rooted in the minimization of directed edge structural entropy, presents a promising avenue for extracting hierarchical structures from the network with intricate topology. To clearly define a greedy partition tree construction algorithm, we introduce the following meta-operations in Alg. 1.
These meta- veta-operations collectively define the intricate logic underlying the greedy partition tree construction algorithm, providing a comprehensive framework for constructing hierarchical structures in graph data while adhering to the principles of minimizing directed edge structural entropy. Building upon these foundations, we employ meta-operations to present the detailed workflow of the greedy structural tree construction algorithm. This facilitates the goarse-grained HKT construction from topological perspective, ultimately achieving digraph data knowledge discovery (i.e., Step Knowledge Discovery (a) in our proposed EDEN as illustrated in Fig. 2).
The Alg. outlines the construction of a heitht-l partition tree algorithm, emphasizing the minimization of directed structural uncertainty. It begins by sorting input data in non-decreasing order. Subsequently, it constructs an initial partition tree, using a greedy approach that iteratively combines nodes until the root has only two children. After that, enters a phase of height reduction, wherein nodes contributing to excess height detached iteratively until the tree attains height h. To stabilize the structure, it inserts filler nodes for any node with height discrepancy exceeding 1. This three-phase process ensures the efficient construction of a height-limited partition tree while minimizing directed structural measurement.
A.6. The Proof of Theorem 3.1
As discussed in Sec. 3.1, node profiles in a digraph act essential identifiers. These profiles not only instrumental in distinguishing nodes but also play a critical role in the construction of data knowledge. Recognizing this, our proposed partition-based node MI neural estimation seeks to further refine the coarse-grained HKT, which is initialized by the greedy algorithm. This refinement is achieved by quantifying the correlations between node profiles within the partition tree, thereby enhancing the granularity of the HKT. The refined tree provides a more accurate and nuanced representation of the graph, laying a robust foundation for subsequent KD. This process ensures that both topological structure and node profile information are effectively leveraged in the distillation, leading to improved model performance.
17

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Algorithm Meta-operation (Function) Definitions
Definition, node ea is the root of T, nodes (vi, vt) are child nodes of node vx
// Meta-1:
Counts the number of child nodes of the given node 2/.
Function CountChildren(vs))
Return Number of children of node À
Meta-2: Inserts a new node between nodes vy and vy. with ux as the root.
Function Combine(w;, ut):
Insert new node on between nodes o; by and node ex children +-
on chilhren + 24
children f- f'y
Meta-3: Chooses two nodes (0,, 0j) from 2n childreen maximize the reduction of 7.7 (g). Function PickTwo(G)):
(1,,a)(w.eg)) HT (G) HG)
H Combima(u (G)}
Return (vx, kt)
Meta-4: Computes the height of the partition tree Function TreeHeight(T)): Return h(? (TT
// Meta-5: Detaches node on from the tree T and merges its children to chiildee. Function Detach(u;):
Detach v; from T and merge its children to cy.chldeeen Uj.children t u..children chihldeeen
Delete 1;
// Meta-6: Chooses one node from T based on minimizing the increase of H² (g). Function ChooseNode(T):
argming,
H (m (v.) (g) - (T(9)) v Fr}
Return
// Meta-7:
Computes the absolute difference in height between the parent of o and uy
Function DeltaHeighh(r;)::
Return TrreHeigght (w; -parend) TreeHeight (v:)
Meta-8: Inserts filler node between nodes vi and by keep the tree height balanced.
Function InsertFillerNode(e;;, Uz):
Insert new node vn between nodes and ui
Un children + 2 Dj childdeen t Un
18

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Algorithm 2 Construction of Height-Limited Partition Tree
Input: data 24, size m repeat
Initialize noChange true.
for i = 1 to
do
if z; > Zi+1 then
Swap and 1++1 noChange = false
end if end for
until noChange true
Input: digraph G (V, an integer h 2
Initialize partition tree T with root node À and set all V as leaves
Phase I: Build partition tree from leaves to root, using the greedy method while CountChildren(( (r) > 2 do
(ee, 0/) <- PickTwo(()
Combine(w;, 03)
end while
Phase II: Height reduction to h while TreeHeight ) > h do
0/ + ChooseNode G, V*)
(wt from T
end while
Phase III: Stabilize tree structure for Each U; 'T do if DeltaHeight((:)) > then InsertFillerNode (e,, 2 varentt end if end for
Considering a digraph g = EV,E) and its coarse- greenerrindd partition tree T, where V encompasses all nodes in the digraph, along with the corresponding feature and label matrix represented as X and Y. For current partition X, given by T, we employ a sampling strategy to obtain a candidate node subset Qp with K, nodes from the current partition X, and other partitions Xg. Notably, different partitions used for sampling should be at the same height within the HKT (e.g., the current partition X, and other partitions X, should satisfy h(x)) h(x))), Building upon this, reduce the computational complexity, adopt computation-friendl sampling strategy. Specifically, considering the number of nodes in the current partition is 1Xpl. we include all of them in the candidate set Spp Additionally, we perform random sampling for partition by-partition until the total non-duplicated nodes in the 9p satisfy |/pl,, where 2 is used to control the knowledge domain expansion come from the other partitions Xg- This subset 2p is used to generate knowledge that represents the current partition Xy, formally represented as the parent representation this partition in the HKT. Notably, we assign distinct identifiers to the sampled nodes based on their partition affiliations, denoting them as u E X, and Xg- providing clarity in illustrating our method and derivation process.
Building upon this foundation, given the node feature when randomly selecting node from 2, within the current partition Xp. Then, the probability distribution of fo is formally defined as Py. = P(f. Vt E 22p n Xp. Similarly, we can generalize Pt. to scenarios originating from other partitions to obtain P/.. = P(f. = X.)) Vu E 2, n NX In 2pp the definition the generalized neighborhoods for any node is closely tied to the partition provided by the HKT, rather than relying on the traditional definition based on the adjacency matrix A from directed edge sets E.
example, random variable fo is introduced represent the node
19

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Specifically, for nodes belonging to the current partition, denoted as E Xp. their generalized neighborhoods are defined as NoT Xp- This is done identify nodes with sufficient information to efficiently represent the current partition i.e., (measure MI between and (n?). ). As for nodes belonging to other partitions, denoted as u E Xx their generalized neighborhoods are defined as NI. = Xp U Kg. This is intended to address the limitations of the coarse- saause-raiind partition tree produced by considering only topological metrics. In other words, we aim to identify sets of nodes within other partitions that effectively capture the representation of both the current partition X, (explore potential correlation from the profile perspective) and their own partition X, (inherit their own partition criteria about directed structural information measurement), thereby refining the HKT through MI measurement between u and NT.
Notably, chose No X2 for the following reasons: (1) We aim to calculate the MI neural estimation between the current node u and its generalized neighborhoods X₂ as a criterion for quantifying affinity scores. This approach ensures that nodes representative of the current partition receive higher affinity scores. Therefore, the generalized neighborhood of the current node needs to be closely related to the partition to which the node belongs, leading us to impose this restriction rather than defining the neighborhood as all nodes V. For more on the motivation, intuition, and theory behind this mechanism, please refer to Sec. 2.2. As for the details on the calculation of affinity scores, we recommend referring to Sec. 3.2 on knowledge generation. (2) In general, the number of partitions X, is considerably smaller than the total set of nodes V. As a result, one of the key motivations for imposing this neighborhood restriction is to minimize computational overhead and improve overall runtime efficiency. By limiting the scope of the calculations, we are able to streamline the process without sacrificing performance, making the method more scalable for large-scale graphs. In summary, expanding the neighborhood to include all nodes would result in higher computational costs and poorer performance. Therefore, we restrict the definition of the generalized neighborhood based on the partition obtained by HKT.
In either case, the generalized neighborhoods are subgraphs containing nodes from V. These nodes may not be directly connected in the original topology but reveal inherent correlations at higher level through the measurement of directed structural information. Therefore, this representation transcends the topological exploration of the digraph by A and reflects intrinsic knowledge at a higher level. Building upon this, considering node v as an example, let fN? be random variable representing the generalized neighborhood feature selected from 2p, originating from the current partition Xp.. We define the probability distribution of JNT as PiwT = P(JNT= = XNy).
Therefore, considering node E Xp example, we define the joint distribution of the random variables of node features and its generalized neighborhood features within partition Xp given by HKT, which is formulated as:
P (Jo: = (f. = X.,, INy = XNT) Ve E Sp Xpp
(13)
where the joint distribution reflects the probability that we randomly pick the corresponding node feature and its generalized neighborhood feature of the same node within partition Xp together. Building upon this, the MI between the node features and the generalized neighborhood features within the current partition Xp is defined as the KL-divergence between the joint
distribution P (fer fNT) and the product of the marginal distributions of the two random variables Pj. @ PINT The above process can be formally defined as:
I (NoS = DKr. (P IPr, @ P.wE)
(14)
This MI measures the mutual dependency between the selected node and its generalized neighborhoods in 2p. The KL divergence adopts the f frrepresentation (Belghazi et al., 2018) is defined as:
DKL (P (fT IPP @ @PINT) 2 sup FEF
E x. X NT ~ XN (fonNNT)) ( [F (x.xxNz'
(15)
E Ex.~Ps XNI PIN E (x(xxXXxT)
F PINT
sup FEF
where F is an arbitrary class of functions that maps pair of selected node features and its generalized neighborhood features to a real value. Here, we use --,-) to compute the dependency. If we explore any possible function F' E F, it can serve as a tight lower bound for MI. Building upon this, we can naturally extend the above derivation process to the scenario of sampling nodes belonging to other partitions, specifically u E 2p n Xg At this point, we can assess the shared contribution of nodes a and u with different affiliations in generating knowledge for the current partition Xp.
20

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
A.7. The Proof of Theorem 3.2
The primary objective here is introduce a node selection criterion that is grounded in quantifying the dependency between the selected node and its generalized neighborhoods. This dependency serves as the foundation for assessing the relevance and influence of cach node within its local structure. The key insights behind using this dependency as guiding principle are central to the formulation of the criterion function. By leveraging this approach, we aim to enhance the process of knowledge generation for the current partition Xz. ensuring that both local and global relationships are effectively captured and utilized in the knowledge distillation process. The detailed reasoning and benefits of this approach are outlined as follows:
(1) In our definition, the generalized neighborhoods of the selected node are closely tied to the current partition X, and their own partition X; Thus, measuring this dependency is equivalent to quantifying the correlation between the representation of the selected node and the knowledge possessed by the current partition and their own partition.
(2) The node-selection criterion is essentially mechanism for weight allocation. Since the candidate node set is fixed by the sampling process, this step aims to assign higher affinity scores to nodes that better represent the current and their own partition. This guides the knowledge generation process to acquire the parent node representation for the current partition.
Building upon this, instead of calculating the exact MI based on KL divergence. we opt for non-KL divergences to offer favorable flexibility and optimization convenience. Remarkably, both non-KL. and KL divergences can be formulated within the same f-representation framework. We commence with the general f-divergence between the joint distribution and the product of marginal distributions of vertices and neighborhoods. The above process can be formally defined as follows:
P(fe,tNT) PI.PETT
Dy (P (fo &PWT)) -
dX.T
(16)
where f(.) represents a convex and sowr--scmioontiuuuus divergence function. When f(z) = x log the f-divergence is specified as the Kullback-Leibler (KL) divergence. The function f()) has a convex conjugate function, denoted as f*), where (1) = supzedom, (tr - [x)), and domy is the domain of f(-). It's important to note that these two functions, f()) and (t), are dual to each other. According to the Fenchel conjugate (Hiriart-Urruty Lemaréchal, 2004) and node sampling space S2p based on different affiliations given by HKT, the f-divergence can be modified as:
Dj (P [P @ PINT))
-| PXPINT sup tEdomy.
P PX. - m10)
PI.PINT
(17)
2 sup E FEJ (((oSN
[F (X.:XNT)
PIN [5t (F (Xmm
where F represents any function that maps the selected node and its generalized neighborhood features to scalar, and the function F(--) serves variational representation of t. is a randomly selected node from Spp excluding This step confines the quantification of MI to the sampling space of 2pp providing finer-grained quantification criterion. Additionally, we employ an activation function ø : R + domy. to constrain the function value F(:..) + -F). Thus, we obtain:
|p 69 PINT)) Dj (fr.
(18)
E BP((tntN))) 1o (F (X,, XxT))) EPj. [1r (a (xxm XXT FEF
(x.:
sup
Given that (t))) also belongs F and its value falls within domy-, the optimal solution satisfies the equation. Assuming the divergence function is [(x) x log the conjugate divergence function is f()() exx((1 - 1). and the activation function is (zx)= = x, we can derive the the f-representation of KL divergence shown in Eq. (15). It is important to note that the choice of the activation function (.) is not unique, and our target is to identify one that facilitates both derivation and computation. Here, we explore an alternative form of divergence utilizing of f-representation, known as GAN-like divergence.
In this context, we employ a specific form of the divergence function, given by f(x) = x log x - (x + 1) log(2: æ + 1), with
21

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
the conjugated divergence function defined as [((()= - log(1 - exp(()) (Nowozin al., 2016). The chosen activation function is (()) log(1 exp(.)). The GAN-like divergence can be expressed as:
DGAN
P (fT. p| @ PINT))
B PP(FodNTT) (or (Xo, XNT))] Rp, Pr. 5r (o (F(F (x., X.TXNNT
2 sup FEF
E Pr(FentNT) [-log log (1 exp (-o (X,; X N
sup FEF
1 (1-exp) -llg
1 1+e F (X.X
sup FEF
E EPiiiPNyT log
1
(19)
=sup FEF
R P(IINNNT) log
1 -5 -F(XxXXN2
1
+ sup PEF
PI log
1
Ite FXxxIXxT))
B P((FONNT) [logo (F (P,xxxXN!) XNT))]}
sup FEF
E BP NN log (1- ( (xx::NNN)))
sup PEF
where, (-) denotes the sigmoid function. Ultimately, the GAN-like divergence transforms the f-divergence into binary cross-entropy, akin to the objective function used for training the discriminator in GAN (Goodfellow et al., 2014). In the aforementioned process of selecting sub-nodes suitable for generating knowledge for the current partition X,, the above lower bound consists of two components. The first term assesses the effective representational capability of the selected node for its generalized neighborhoods. Considering the close correlation of the definition of generalized neighborhoods with the current partition, it can be regarded as a measure from the embedding perspective of the relevance of the selected node the knowledge of the current partition. The second term binds the measurement space of relevance with the sampling space based the affiliation relationship. It gauges the expressive capability of the currently selected node for partition knowledge compared to other nodes in the sampling set. Based on the aforementioned inference, we can generalize it to nodes u belonging to other partitions Xq-
A.8. The Proof of Theorem 3.3
To determine the form of the function -.,,,), we parametrize F(;,.) using trainable neural networks instead of manual design. The parameterized function is denoted as (t.,,,, where w generally represents the trainable parameters. In this study, (:..) :) has two construction mechanisms based on the partition X; to which the selected node belongs and the current partition X, where the knowledge generation process is applied. The criteria are as follows:
(1) Intra-partition: Identifying nodes v that efficiently represent the current partition Xp (i.e., MI between X, and X X,) and assigning them higher affinity scores to dominate the weighted knowledge generation process based the spp.
(2) Inter-partition:
Identifying nodes within other partitions X, that potentially represent the current partition effectively (i.e.. MI between X, and X²). Meanwhile, node is required to adhere to well-defined criteria for directed structural information measurement inherited from its corresponding partition to ensure accuracy (i.e., MI between X,, and Xz). Building upon this foundation, we achieve MI neural estimation between X,, and XNT = X, U Xg to obtain efficient affinity scores for u E Xy. These nodes might not have been correctly assigned to the current partition X, initially due to coarse-grained directed structural measurements.
Following these criteria, reformulate the problem into fine finennrained selection task for nodes contained within two partition roles Xr, and X,,- Building on this, we provide the instantiation of the criterion function C(-), incorporating (1) model-agnostic digraph learning function M executed at each tree layer of the HKT, which can leverage some widely used model architectures such as DiGCN (Tong et al., 2020a), MagNet (Zhang et al., 2021c), HoloNet (Koke & Cremers, 2024), or be tailored for practical settings; (2) mapping functions W1 and W2 dedicated to encoding the currently selected node and its generalized neighborhoods, respectively; (3) two functions Qintra and Qinter for generating the final affinity scores
22

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
based on the encoding results and the current node's partition affiliation. Furthermore, efficiently encode the generalized neighborhoods, we perform 1-step label propagation based on the high-level neighborhood relations Tx, in partition A, provided by the HKT. The above process based on the current partition Xp can be formally defined as
Fintre- =
Qintra (WiM (X.)..) W2 (M (M(XxTT)) ))),
pinter = Qinter (Wi (M We (M (XNT)))
XAT= = Agg X Vi E EX))
XNT = Agg
{ Vi E X, X.).
(20)
X x + (1-T)
Σ
1
_
vie X, or i E X2 UN.
jeTx, or UTXx, Vd.d,
We adopt the approximate calculation method for the personalized PageRank (Kliepera al., 2019). Meanwhile, we set 0.5 and I = 5 by default to capture deep structural information. Due to the small-world phenomenon, aim traverse as many nodes as possible within the subgraph through such settings. Moreover, Agg(+) is generalized neighborhood representation aggregation function. This function can be implemented through weight-free operations. It is noteworthy that, due to the shared encoding function weights within each partition X,, the results generated by the neighborhood representation function in partitions with different node quantities must have the same size. In our implementation, considering computational costs, we default to using the weight-free form.
In this manner, the parameterized GAN-like divergence serves as a variational lower bound for the theoretical GAN-like- divergence-basee MI between digraph nodes and their generalized neighborhoods. Taking the node belonging to the current partition Xx as example, we obtain the following representation. Similarly, an extension can be applied to nodes belonging to other partitions X..
I(0) (Sor (NTT)
= DGAN
P (for lp @ xPITT) 2 ((0)
(fur
E (P((nfAt)) [logo (F (NT)1]
max
Ep EPIoPPIN [log (1- (1 (x::
(21)
+ maxx
log a (Fir XNT))
max 1
log (1-o Fww (xv XNI)) )))
max
22
(v.B)ER
A.9. Algorithm Complexity Analysis
The complexity of Step 1 is O (h(m log + n)). Notably, as T tends to be balanced during the structural measurement minimization, height h is approximately log Additionally, considering that m >> n, the complexity of Step 1 scales nearly linearly with the number of edges. Subsequently, Step 2 and Step 3 introduce the KD-based training framework. Considering L-layer MLP and HKT layer-wise DiGNN, the time complexity can be bound by O(h(mm/ + Lkn log ne²)) )). In comparison to Step 1, it is negligible. This is attributed to the random walk and feature transformation can be executed with significantly lower costs due to sparse matrices and parallelism in computation. Moreover, in practice, we can employ a lightweight HKT layer-wise digraph learning to achieve acceleration. Consequently, ((m) in Step I remains the primary bottleneck for achieving scalability.
A. Dataset Description
We evaluate the performance of our proposed EDEN on 10 digraph and 4 undirected graph benchmark datasets, considering the neve- transductive/idductive supi-supe classification task and three link-level prediction tasks. The 10 publicly partitioned digraph datasets include 3 citation networks (CoraML, Citeseer, and ogbn-arxiv) in (Bojchevski & Günnemann, 2018 8; Hu et al., 2020), social networks (Slashdot and Epinions) in (Ordozgoiti al., 2020; Massa & Avesani,
23

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Table? 7. The statistical information of the experimental di(graph) benchmark datasets.
Topology- Prooilee
Datasets
#Node
#Fearrres
#idges
#N Classes
#N Valis/Val/esst
Train/Val/Tes
TTisk
Deseription
Homophily
Photo
7,487
745
119,043
8
2/9,81
Undirected
Transdactive
Co-purchase
Homopbily
Computers
13.381
767
245.778
10
1100/1100/100
Undirected
Trunschctive
Co-purchase
Hiterophilyy
PPI
56,914
50
818.716
121
4,555H4555539, 993
Undirected
Inductive
Pretein
Heterophily
Flickr
89,250
500
899,756
111407,140007,,00
Undirected
Inductive
Image
Homoptily
ConMMI
2,595
2,879
8,416
140/500/3355
Node&
Citation
Homophilly
CieSeer
3,312
3,703
4,591
6
120/500X26922
80%6/199//5
Node&Ling
Chaation
Homophily
WikiCS
11,701
300
290,519
10
580/1769/8847
80%6/15%//5
Node&Link
Weblink
Homophily
Tolokens
11,758
10
519,000
4046/2599/25%
80%//199/59
NodeRLing
Crowd-sourcing
Heterophily
Empire
22,662
300
32,927
50%/255%/255
80///15%
Node&Link
Article Syntax.
Heterophily
Rating
24,492
300
93.050
5
50% 4095/255
80%//15%%/5%
Node&Link
Rating
Heterophily
Arxiv
169,343
2,315,598
60%9/20%/20%
80%//15%//5
Node&Link
Citation
Slachdor
75,144
100
425,702
8056/1 (199//5
Link Only
Social
Epinions
114,467
717.129
8959/1595/559
Link Only
Social
WikiTalk
2,388,953
100
5,018,445
8056/1599/54
Link Only
Co-editor
2005), web-link network (WikiCS) in (Mernyei & Cangea, 2020), crowd-sourcing network (Toloklers) (Platonov et al., 2023), syntax network (Empire), rating network (Rating) (Platonov et al., 2023), and co-editor network (Leskovec et al., 2010). In the transductive scenario, we conduct experiments on two co-purchase networks. In the inductive scenario, we perform experiments on the image relation and the protein interaction networks. The dataset statistics are shown in Table 7 and more descriptions can found later in this section.
We need to clarify that we are using the directed version of the dataset instead of the one provided by the PyG library (CoraML, CiteSeer)', WikiCS paper* and the raw data given by the OGB (ogb-arxiv)². Meanwhile, remove the redundant multiple and self-loop edges to further normalize the 10 digraph datasets. In addition, for Slashdot, Epinions, and WikiTalk, the PyGSD (He et al., 2023) library reveals only the topology and lacks the corresponding node features and labels. Therefore, we generate the node features using eigenvectors of the regularised topology. Building upon this foundation, the description of all digraph benchmark datasets is listed below:
Photo and Computers (Shchur al., 2018) are segments of the Amazon co-purchase graph. Nodes represent goods and edges represent that two goods are frequently bought together. Given product reviews as bag-of-words node features, the task is to map goods to their respective product category.
PPI (Zeng et al., 2020) stands for Protein-Proteii Interaction (PPI) network, where nodes represent protein. If two proteins participate in life process or perform a certain function together, it is regarded as an interaction between these two proteins. Complex interactions between multiple proteins can be described by PPI networks.
Flickr (Zeng et al., 2020) dataset originates from the SNAP, they collect Flickr data and generate an undirected graph. Nodes represent images, and edges connect images with common properties like geographic location, gallery, or shared comments. Node fcatures are 500-dimensional ong-of-wordh representations extracted from the images. The labels are manually merged from the 81 tags into 7 classes.
CoraML and CiteSeer (Bojchevski & Glinnemann, 2018) are three citation network datasets. In these three networks, papers from different topics are considered nodes, and the edges are citations among the papers. The node attributes are binary word vectors, and class labels are the topics the papers belong to.
WikiCS (Memyyi & Cangea, 2020) is Wikipedia- based dataset for bench-marking GNNs. The dataset consists of nodes corresponding to computer science articles, with edges based hyperlinks and 10 classes representing different branches of the field. The node features are derived from the text of the corresponding articles. They were calculated as the average of pre-trained GloVe word embeddings (Pennington et al.. 2014), resulting in 300-dimensional node features.
Tolokers (Platonov et al., 2023) is derived from the Toloka crowdsourcing platform (Likhobaba et al., 2023). Nodes correspond to tolokers (workers) who have engaged in at least one of the 13 selected projects. An edge connects two tolokers if they have collaborated on the same task. The objective is to predict which tolokers have been banned in one of the projects Node features are derived from the worker's profile information and task performance statistics.
24

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Empire (Platonov et al., 2023) is based on the Roman Empire article from the English Wikipedia (Lhoest et al., 2021), each node in the graph corresponds an edge if the words either follow each other in the text or are linked in the sentence's dependency tree. Thus, the graph represents a chain graph with additional connections.
non-unique word in the text, mirroring the article's length. Nodes are connected by
Rating (Platonov et al., 2023) is derived from the Amazon product so-purchasing network metadata available in the SNAP* datasets (Leskovec & Krevl, 2014). Nodes represent various products, and edges connect items frequently bought together. The task involves predicting the average rating given by reviewers, categorized into five classes. Node features are based on the mean FastText embeddings (Grave et al., 2018) of words in the product description. To manage graph size, only the largest connected component of the 5-core is considered.
ogbn-arxiv (Hu et al., 2020) is citation graphs indexed by MAG (Wang et al., 2020). Each paper comes with 128- dimensional feature vector obtained by averaging the embeddings of words in its title and abstract. The embeddings of individual words are computed by running the skip-gram model.
Slashdot (Ordozgoiti et al., 2020) is from a technology-relatee news wehsite with user communities. The website introduced Slashdot Zoo features that allow users to tag each other as friends or foes. The dataset is a common signed social network with friends and enemies labels. In our experiments, only consider friendships.
Epinions (Massa & Avesani, 2005) is a who-trust-whom online social network. Members of the site can indicate their trust or distrust of the reviews of others. The network reflects people's ssples opinions of others. In our experiments, we only consider the "trust" relationships.
WikiTalk (Leskovec et al., 2010) includes all users and discussions from the inception of Wikipedia. The network comprises n 2, 388, 953 nodes, where each node represents a Wikipedia user, and directed edge from node U, node vy indicates that user i edited user j 's talk page at least once. For our analysis, we extract the largest weakly connected component.
A.11. Compared Baselines
The baselines we employ are as follows: (1) Directed spatial-based approaches: DGCN (Tong et al., 2020b), DIMPA (He et al., 2022b), NSTE (Kollias et al., 2022), D-HYPR (Zhou et al., 2022), and Dir-GNN (Rossi al., 2023); (2) Directed resctral-based approaches: DiGCN (Tong et al., 2020a), MagNet (Zhang al., 2021c). MGC(Zhang et al., 2021a), and HoloNet (Koke & Cremers, 2024). Furthermore, to verify the generalization of our proposed EDEN, we compare the undirected GNNs in digraphs with coarse undirected transformation (i.e., convert directed edges into undirected edges): GCN (Kipf& & Welling, 2017), GAT (Veličković et al., 2018). GCNII (Chen et al., 2020), GATv2 (Brody et al., 2022), OptBasisGNN (Guo& & Wei, 2023) (OptBG), NAGphormer (Chen et al., 2023) (NAG), and AGT (Ma et al., 2023). The descriptions of them can be found later this section. For link-level dataset split, we are aligned with previous work (Zhang al., 2021c; He al., 2022a; 2023). To alleviate the influence of randomness, we repeat each experiment 10 times to represent unbiased performance and running time (second report). Notably, we present experiment results with various baselines in separate modules, avoiding abundant charts and validating the generalizability of EDEN. To ensure fair experimental comparison and avoid introducing excessive trainable parameters, we fix the random walk length to 5 in HKT-based leaf prediction. This setting does not treat walk length as a hyperparameter for optimal search.
Notably, EDEN can be regarded as a novel digraph learning paradigm or a hot-and-plug online distillation module for prevalent (Di)GNNs. Now, we elaborate on their experimental implementations.
(1) A new digraph learning paradigm: Different from the direct application of existing DiGNNs, in the HKT layer-wise distillation process based on HKT, we implement the digraph learning functions of (7) and (8) through personalized model design. Specifically, to reduce computational costs, we employ the magnetic Laplacian proposed in MagNet (Zhang et al., 2021c) for digraph convolution. Compared to MagNet, EDEN pre-computes T. iterations of feature propagation and compresses complex learning processes into simple linear mappings, maximizing training and inference efficiency. Building upon this, a personalized model design for the online distillation process is implemented to achieve end-to-end training.
(2) A hot-and-plue online distillation module: Essentially, EDEN serves as general online distillation framework, introducing hierarchical knowledge transfer mechanism for existing DiGNNs. In other words, EDEN seamlessly integrates into the HKT layer-wise digraph learning functions (i.e., utilize existing digraph neural architectures as digraph learning function in (7) and (8) to generate node embeddings or soft labels) to improve predictions.
hhttps//ssappssanfor..co
25

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
DGCN (Tong al., 2020b): DGCN proposes the first and second-order proximity of neighbors design new message- passing mechanism, which in turn learns aggregators based incoming and outgoing edges using two sets of independent learnable parameters.
DIMPA (He et al., 2022b): DIMPA represents source and target nodes separately. However, DIMPA aggregates the neighborhood information within K hops in each layer to further increase the receptive field (RF), and it performs a weighted average of the multi-hop neighborhood information to capture the local network information.
NSTE (Kollias et al.. 2022): NSTE is inspired by the 1-WL graph isomorphism test, which uses two sets of trainable weights to encode source and target nodes separately. Then, the information aggregation weights are tuned based on the parameterized feature propagation process to generate node representations.
D-HYPR (Zhou et al., 2022): D-HYPR introduces hyperbolic collaborative learning from diverse neighborhoods and regularizers. This conceptually simple yet effective framework extends seamlessly to digraphs with cycles and non-transitive relations, showcasing versatility in various downstream tasks.
incorporates socio- cecioosscchooogicll-nnpppe
Dir-GNN (Rossi et al., 2023): Dir-GNN introduces a versatile framework tailored for heterophilous settings. It addresses edge directionality by conducting separate aggregations of incoming and outgoing edges. Demonstrated to match the expressivity of the directed WL test, Dir-GNN outperforms conventional MPNNs in accurately modeling digraphs.
DiGCN (Tong et al., 2020a): DiGCN notices the inherent connections between graph Laplacian and stationary distributions of PageRank, it theoretically extends personalized PageRank to construct real symmetric Digraph Laplacian. Meanwhile, DiGCN uses first-order and second-order neighbor proximity to further increase RF.
MagNet (Zhang et al., 2021c): MagNet utilizes complex numbers to model directed information, proposes a spectral GNN for digraphs based on a complex Hermitian matrix known as the magnetic Laplacian. Meanwhile, Mag uses additional trainable parameters to combine the real and imaginary filter signals separately to achieve better prediction performance.
MGC (Zhang et al., 2021a): MGC introduces the magnetic Laplacian, a discrete operator with the magnetic field, which preserves edge directionality by encoding it into a complex phase with an electric charge parameter. By adopting a truncated variant of PageRank, it designs and builds a low-pass filter for homogeneous graphs and a high-pass filter for heterogeneous graphs.
HoloNet demonstrates that spectral convolution can extend to digraphs. By leveraging advanced tools from complex analysis and spectral theory, HoloNet introduces spectral convolutions tailored for digraphs.
HoloNet (Koke& & Cremers, 2024):
GCN (Kipf& & Welling, 2017): GCN is guided by a localized first-order approximation of spectral graph convolutions. This model's 's scalability is directly proportional to the number of edges, and it learns intermediate representations in hidden layers that capture both the structure and node features.
GCNI (Chen et al., 2020) incorporates initial residual and identity mapping. Theoretical and empirical evidence is presented to demonstrate these techniques alleviate the over-smoothing issue.
GAT (Veličković et al., 2018) utilizes attention mechanisms quantify the importance of neighbors for message aggregation. This strategy enables implicitly specifying different weights to different nodes in neighborhood, without depending on the graph structure upfront.
GATv2 (Brody et al., 2022) introduces a variant with dynamic graph attention mechanisms to improve GAT. This strategy provides better node representation capabilities and enhanced robustness when dealing with graph noise.
OpNBasisGNN (Guo& & Wei, 2023): OptBasisGNN revolutionizes GNNs by redefining polynomial filters. It dynamically learns suitable polynomial bases from training data, addressing fundamental adaptability. OptBasisGNN innovatively addresses the challenge of determining the optimal polynomial basis for specific graph and signal, showcasing its effectiveness in extensive experiments.
NAGphormer (Chen et al., 2023) treats each node as a sequence containing a series of tokens. For each node, NAGphormer aggregates the neighborhood features from different hops into different representations.
AGT (Ma et al., 2023) consists of a learnable centrality encoding strategy and a kenneled local structure encoding mechanism to extract structural patterns from the centrality and subgraph views to improve node representations for the node-level downstream tasks.
26

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
70
70
70
65
60
60
60
AGT
AST
AGT
DIMPA
55
DIMPA
DIMPA
50
Dir-GNN
DNN
50
Dir-GNN
MagNet
50
MagNet
MagNet
HoloNet
NoloNet
40
HoloNet
40
EDEN
45
EDEN
EDEN
50
100
150
200
50
100
150
200
50
100
150
200
Training Epoch (ACC)
Training Epoch (ACC)
Training Epoch (ACC)
(a) CiteSeer-Node-C
(b) CiteSeer-Link-C
(c) Arxiv-Node-C
80
90
90
70
AGT
85
AGT
AGT
DIMPA
DIMPA
DIMPA
Dir-GNN
80
Dir-GNN
Dir-GNN
60
MagNet
MagNet
80
Mag MagNet Net
HoloNet
75
HoloNet
HoloNet
EDEN
EDEN
EDEN
50
100
150
200
50
100
150
200
0
50
100
150
200
Training Epoch (ACC)
Training Epoch (AUC)
Training Epoch (AUC)
(d) Arxiv-Link-C
(c) Slashdot-Exist
(f) Slashdot-Direct
Figure 6. Convergence curves on node- and link-level tasks.
A.12. Hyperparameter settings
The hyperparameters in the baseline models set according to the original paper if available. Otherwise, we perform hyperparameter search via the Optuna (Akiba et al., 2019). For proposed EDEN, during the topology- gaarse-graieed HKT construction, we perform grid search in the interval [3. 10) to determine the height of HKT. In the profile-oriented fine-grained HKT correction, a grid search is conducted in the interval [1, 2] to obtain the optimal k, deciding the knowledge reception field when generating parent node representations for the current partition. For random walk-based leaf prediction, we search in the interval [o, 1] based on node-level or link-level downstream tasks to determine the optimal walking probability. Additionally, within the same interval, we search determine the hyperparameter a for knowledge distillation loss, ensuring optimal convergence.
A.13. Experiment Environment
The experiments are conducted on Intel(R) Xeon(R) Gold 6230R CPU @ 2.10GHz, NVIDIA GeForce RTX 3090 with 24GB memory, and CUDA 1.8. The operating system is Ubuntu 18.04.6 with 768GB of memory. As for software versions use Python 3.9 and Pytorch 1.11.0.
A.14. Extend Experimental Results
Convergence Analysis. To supplement answer Q3, we first present the convergence curves in Fig. 6, where we observe that EDEN exhibits higher initial performance and more stable convergence. For instance, in the Node-C for the CiteSeer, EDEN nearly reaches converged performance by the 25th epoch and maintains stability throughout the subsequent training process. Notably, various link-level downstream tasks, benefiting from larger number of training samples, exhibit smoother optimization curves and more stable predictive performances compared to node-level classification tasks.
Hyperparameter Analysis. To provide comprehensive analysis of the robustness EDEN from the perspective of hyperparameter sensitivity, we supplement the experimental results in Fig.7 7 with the outcomes of HKT-based random walk
27

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
83
95
92
81.
78
02
30 of 07 as
03 04 oP
62 04 0
0g 2g Rgdy
ag felts agarne
Prw
07
Prw
Prw
Prw
Syw
Srw
Srw
Srw
(a) Node-C (ACC)
(b) Link-C (ACC)
(c) Link-E (AUC)
(d) Link-D (AUC)
Figure 7. The sensitive analysis based random walk under Tolokers.
61.0
90.0
Test Accuracy (%)
EDEN
EDEN
EDEN
EDEN
81.5
Text Accuracy
60.5
93.5
89.5
TestAUC
Test TestAUC
81.0
60.0
93.0
89.0
59.5
88.5
80.0
59.0
0.10.20,3044..00 0.36644...000
92.0
0.10.20,3040..
88.0
0.10.20.30.00.
0...
KD Align Parameter
KD Align
K1 Align
(a) Empire-Node-C
(b) Empire-Link-C
(c) Epinions-Linee Links
(d) Epinions- LinkkD
Figure The sensitive analysis of KD loss factor.
sampling for leaf-centric prediction, considering various probabilities of transitioning between different identity nodes (i.e., parents, siblings, and children). Notably, we do not discuss the sampling probability regarding children separately. This because their main role is to provide return probabilities in the random walk process to yield richer sampling sequences, without explicitly indicating the identity of the next node to visit. Before giving our analysis, first revisit the key insights introduced in Sec.3.3: (1) For node-level downstream tasks, it's preferable to sample the parent of the current leaf node to offer a rich high-level representation of the current label class. (2) For link-level downstream tasks, it's preferable to sample the siblings of the current leaf node to provide topologically relevant contextual insights at the same level. Based on the experimental results, we observe that for Node-C, larger values of Prw and smaller values of syw yield better predictive performance, whereas for the three distinct link-level downstream tasks, smaller values of Prw and larger values of 8 are preferable. This validates our aforementioned assertions and provides empirical reference for selecting hyperparameters when practically applying EDEN.
Additionally, in Fig. 8, we provide insights into how varying the coefficient a in the a-flexible KD loss impacts the optimization process, reflected in the final predictive performance. According to our experimental results, in most cases, EDEN should prioritize the KD process during end-to-end optimization. This because the node-adaptive trainable knowledge generation and transfer processes ensure high-quality KD, thereby positively influencing downstream task predictions. Notably, smaller values of a perform better in edge existence problems. This is because the cross-entropy loss function, used provide supervision, aids significantly in coarser-grained existence problems, while finer-grained issues like directionality and classification often benefit more from data-driven high-quality knowledge. In nutshell, we recommend smaller values for edge existence problems and larger values for other tasks, followed by manual adjustments based on practical performance.
Comprehensive Results.
To present comprehensive experimental findings, this section includes additional results (Table Table 9, Table 10, and Table ??) that couldn't be fully showcased in the main due space limitations. These additional experimental results, consistent with the trends presented in the main text, further substantiate thorough assessment, we introduce two additional evaluation metrics, Area Under Curve (AUC) and Average Precision (AP), alongside the commonly known Accuracy (ACC). We default to using AUC and AP in the evaluation of the link prediction tasks, and ACC to evaluate the predictive performance of node-level tasks. Regarding the experimental results of Dir-GNN and HoloNet on the Empire dataset, we would like to clarify that we ensured fair comparison by using a class-bald balanced dataset split instead of the pre-split datasets used in Dir-GNI and HoloNet.
claims in Sec. 4. Notably, to provide more
28

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Table
Test accuracy (%) in directed Node-C.
Models
CoraML
CiteSeer
WikiCS
Tolokers
Empire
Rating
Arxiv
GCN
80.6+0.4
62.1
78.3+0.2
78.0+0.1
75.8+0.5
42.5+0.4
65.220.2
GAT
80.7+0.6
62.6#0.6
78.2+0.3
78.4+0.2
77. 77.8+0.8
42.9+0.5
65.9#0.3
OptBG
81.0+0.5
63.2+0.4
78.5+0.2
78.6+0.2
78.0#0.6
43.240.4
66.3+0.3
NAG
81.4+0.7
62.7=0.5
78.6+0.3
78.4+0.4
77.5=0.9
43.1+0.6
66.5+0.4
NSTE
82.2+0.5
64.320.7
79.0+0.3
79.3+0.3
78.920.6
44.7+0.6
67.220.4
Dir-GNN
82.6+0.6
64.0#0.6
79.1#0.4
79.1#0.3
79.1
45.0+0.5
67.4 67.420.3
MGC
82.3+0.4
63.9+0.5
78.8+0.2
79.0+0.2
78.640.4
44.8+0.4
67.0+0.2
EDEN
84.6+0.5
65.820.6
81.4+0.3
81.3+0.2
81.1+0.6
46.3+0.4
69.7.0.3
Table 9, Model performance (%) three directed link-level downstream tasks.
Datasets (-)
Slashdot
Epinions
Tasks (-))
Exist
Direct
Link-C
Exist
Direct
Link-C
Models (1)
AUC
AUC
ACC
AUC
AUC
ACC
GCNII
88.620.1
88.4+0.0
90.3+0.1
90.4+0.1
84.0=0.1
91.3+0.1
91.3+0.0
85.9a0.2
86.3+0.1
82.7.0.1
GATv2
88.2+0.2
88.5+0.1
90.6+0.1
90.4+0.1
83.7+0.3
91.8+0.2
91.640.1
85.5+0.1
85.9+0.1
83.020.2
AGT
88.7=0.2
88.6+0.1
90.1+0.0
90.5+0.1
83.8+0.2
91.5+0.2
91.4+0.2
85.7#0.2
86.2+0.2
83.4+0.1
DGCN
90.3+0.1
90.1+0.0
92.2+0.1
92.4+0.1
85.5+0.2
92.2+0.1
92.5+0.0
87.8+0.1
87.5+0.2
83,6#0.2
DIMPA
90.5+0.1
90.740.1
92.4+0.2
92.1+0.1
85.620.1
92.5+0.1
92.640.1
87.940.1
88.240.1
83.5+0.1
D-HYPR
90.3+0.0
90.6+0.1
92.2+0.1
91.9+0.0
85.4+0.1
92.8+0.1
92.4+0.1
88.2+0.1
88.3+0.0
83.7=0.2
DiGCN
90.4+0.1
90.540.1
92.1+0.1
92.0+0.1
85.240.1
92.4+0.1
92.720.1
88.020.1
87.840.1
83.620.1
HoloNet
90.2+0.1
90.3+0.0
91.8+0.1
92.0+0.0
85.1
92.6+0.1
92.5+0.0
18.1
88.2+0.0
84.0s0.1
EDEN
91.8+0.1
92.0+0.0
93.3+0.1
93.1+0.0
87.1+0.2
93.5+0.1
93.7+0.0
89.4
89.8+0.0
85.7.0.1
AUC stands as a comprehensive metric for evaluating binary classification models. Quantifying the area beneath the ROC curve, it provides a global assessment of the model's 's ability to discriminate between positive and negative instances. AUC is particularly valuable in scenarios with imbalanced datasets, as it remains insensitive to variations in class distribution. Its utility extends to model comparison, offering insights into performance variations across different decision thresholds.
AP involves ranking predictions by their confidence scores, typically probabilities, from highest to lowest, and calculating precision and recall at each threshold. These metrics used construct precision-recal eurve, which plots precision values function of recall. AP itself is computed the weighted mean of precision achieved at each threshold, where the weights are the increments in recall from the previous thresholds. This approach allows AP to summarize the area under the precision-recall curve, providing single-figure measure of model performance that encapsulates both the accuracy and the ranking of the positive predictions. Higher AP values indicate a model that not only predicts the positive class accurately but also ranks those predictions highly, thus demonstrating high precision and recall across the board.
29

---

Toward Data-centric Directed Graph Learning: An Entropy-driven Approach
Table 10. Link- ACC and others AUC (%) in three directed link-level downstream tasks.
Datasets
Tasks
GCN
GAT
OptBG
NAG
NSTE
Dir-GNN
MGC
HoloNet
EDEN
Existence
83.26+0.18
83.96*0.25
83.55+0.16
84.32+0.20
87.94+0.18
88. 15.1510.21
87.86+0.20
87.80+0.24
90.8420.19
CoraMI.
Direction
82.73+0.32
84.25=0.54
83.46+0.40
85.39+0.47
90.74+0.54
91.08+0.45
19.10+0.2
89.83+0.57
92.36200.8
Link-C
69.80=0.45
70.67=0.52
70.54=0.60
71.04+0.56
72.79+0.42
73.11+0.49
72.82+0.60
72.74+0.56
75.18=0.54
Existence
75.60+0.34
76,27=0.28
75.85=0.29
76.94=0.40
79.80#0.42
79.65+0.34
79.46+0.29
79.32+0.30
82.2420.37
CiteSeer
Direction
72.32+0.75 73.4640.53
72.9640.68
73.88 73.8840.67
88.3540.68
88 68.6440.57
88.47+0.41
88.7640.48
90.5620.40
Link-C
61.7410.83
62.4640.72
62.29+0.75
62.86+0.65
14.0444.48
64.35a0.43
63.88+0.50
63. 63.9440.36
66.73+0.57
Existence
90.67:0.07
91.1520.14
90.43+0.10
91.08+0.14
91 91.60#0.08
91.38+0.11
91.13+0.05
91.28*0.09
92.8420.12
WikiCS
Direction
85.2610.37 85.61+0.29
85.40:0.32
85.7540.35
87 87.2810.25
87.1240.30
87.3340.17
87.2440.22 26
90.08=0.24
Link-C
78.7140.15
79.0840.19
78.84+0.23
79.4240.22
81.8320.19
81.6740.14
81.47+0.20
81.2640.18
83.45=0.21
Existence
91.90+0.09
92.23+0.14
92.08=0.099
92. 12.1910.11
93.03+0.1
93.48+0.11
93.69#0.10
93.84+0.08
94.93+0.10
Tolokers
Direction
87.6820.13 87.57=0.08
88.28=40111
88.9740.09
89.42+0.10 10
89.6540.08
89.9240.07
89.7640.11
91.52=0.12
Link-C
77.5420.09
77.8520.14 14
78.20=0.11
78.4940.13
80.2840.07
80.46+0.10
80.8340.08
80.5110.12
82.67=0.13
Existence
62.5140.67
62.93=0.81
63. 14.4....7
63.8540.80
66.35+0.35
66.28+0.42
65. 65.99+0.32
65.8640.46
68.81=0.41
Empire
Direction
48.60+0.95
49.77=0.87
49.8 09.83+
30.16+0.84
53.87+0.42
53.94x0.40
53.58+0.37
53.79+0.45
55.60=0.48
Link-C
52.56+0.86
53.0240.99
1.01
13.1241.17
58.69+0.44
58.6240.45
58. 58.09+0.31
58.33+0.35
60.74+0.39
Existence
73.48+0.45
73.95=0.57
73.60+0.52
75.26+0.43
76.91+0.20
77.48+0.29
77.21+0.18
77.12+0.25
79.52=0.27
Rating
Direction
78.5440.32
78.81+0.41
78.90 08.901
79 29.4220.33
82.85+0.27
83.4640.30
33.6810.21
83.30&0.33
19.00.0
Link-C
58.6340.46
58,79 38,79
38.60+0.64
59.1340.37
63.6440.28
64.2340.39
64.28+0.25
64.3240.32
66.37=0.35
Existence
82.04+0.15
81.8720.19
82.24=0.17
82.44+0.16
84.82+0.23
85.37+0.19
84.70+0.28
85.25+0.20
87.2420.23
Arxiv
Direction
88.5640.16
88.71+0.20
88.94#0.21
89. 19.10#0.22
93.3440.14
93.6240.17
93.2740.11
93.40=0.15
94.4820.16
Link-C
74.70+0.17
14.5320.16
74.93=0.20
75.0540.18
78.6340.17
78 78.8940.15
78.70
78.9340.21
80.1620.21
30


Reviews:
[{"review_id": "339iUKssZi", "reviewer": "Reviewer_RNnj", "summary": "The author proposes a complex but effective method called EDEN, which tailored for the directed graph, specifically, it frist build a coarse-gradined Hierarchical Knowledge Tree (HKT), then, it refine the HKT with knowledge flow in the HKT. The method is widely adopted in the 14 graphs and the results valid the method's effectiveness.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Due to the word limit imposed by the new regulations of ICML 2025 rebuttal, we have not provided detailed references, but we will gladly supply them in our subsequent discussions if needed.\n\n**Q1: Relation To Broader Scientific Literature**\n\nI sincerely apologize for any confusion that may have been caused. Please allow us to clarify our motivations and methodology.\n\n**Motivation**\n\nRecent studies have shown that considering the directionality of edges provides a novel perspective to address the long-standing heterogeneity challenges in graph ML and achieves impressive performance. These studies collectively emphasize the necessity of extracting implicit data knowledge from directed graphs to drive the model’s learning\n\n- Dir-GNN [LoG 2024] facilitates homophilous aggregation by **directed edges**.\n- ADPA [ICDE 2024] derives high-order propagation operators from **directed neighboring connectivity**.\n- MAP [WWW 2025] optimizes the strength of directed edges by **node degrees and directed label patterns**.\n\nDespite their effectiveness, the data-centric concept remains unformulated, and optimization perspectives vary. Therefore, we aim to propose a unified framework, which leverages Tree as the carriers of data knowledge and achieves learning. Inspired by Reviewer zpYj Q3, we have conducted additional experiments, which illustrate the superiority of our framework.\n\n**Complexity**\n\nWe acknowledge that the current EDEN may limit scalability. However, this is somewhat inevitable in the early stages of establishing a unified framework. We kindly ask for your understanding in this regard. That being said, we have made practical efforts to simplify EDEN. In Sec 3.4, we introduced a lightweight EDEN, derived from three orthogonal perspectives, and successfully applied it to million-level Arxiv and WikiTalk. Table 1 and Figure 3 indicate that our method demonstrates a certain efficiency advantage.\n\nWe also recognize the need for further efficiency improvements. If possible, we plan to further discuss promising optimization directions in the appendix. For instance, we will explore the possibility of integrating the HKT construction with graph partitioning techniques to enable parallel processing.\n\nWe trust that the above response will address your concerns and enhance your confidence in our manuscript. If you require additional clarification, please do not hesitate to contact me.\n\n**Q2: Other Strengths And Weaknesses**\n\n**Why Tree**\n\n(1) Graph Mining: We adopt the tree structure for its intuitiveness in revealing hierarchical knowledge within structured data. This concept has been theoretically substantiated in prior studies [STOC 1988, TOIT 2016, ICML 2024]. We kindly suggest that you refer to Sec 2.2 for a detailed elaboration.\n\n(2) Graph Learning: In some recent studies [NeurIPS 2021, NeurIPS 2024, ICLR 2024], researchers have abstracted the core of GNN as the tree-based message passing. They have theoretically demonstrated its effectiveness. \n\nBased on these insightful studies, the motivation for adopting a tree-based framework is its versatility and significant potential for data-centric graph learning. Additionally, we optimize the efficiency of this framework in Sec 3.4, rendering it comparable to or even superior to the best baseline shown in Sec 4.\n\n**Why Monte Carlo**\n\nThis technique has recently been widely employed in tree-based numerical computation [SIGMOD 2023, WWW 2024]. In this paper, we construct HKT by minimizing structural entropy via a greedy algorithm, which iteratively selects optimal sets for leaf Combining and Lifting. By integrating the Monte Carlo method, we approximate solutions through random sampling, avoiding exhaustive enumeration of all branches for subsequent greedy selection.\n\n**Real Case**\n\n(1) Research Perspective: As previously mentioned in our response to Q1, the data knowledge derived from directed graphs provides key insights that improve model computation and yield superior performance.\n\n(2) Practical Perspective: Taking citation networks with interdisciplinary (e.g., AI4Science) as an example. In this context, at the first (leaf) level of HKT, we use edge directions to model intra- and inter-field citations. Based on this, HKT endows us with the ability to organize hierarchical knowledge. Specifically, at higher levels of the tree, we can interpret this as progressively more abstract concepts, such as research groups, institutes, and universities. Explicitly representing these concepts can be regarded as revealing data knowledge and providing insights to enhance tree-based graph learning.\n\nWe trust that our response effectively addresses any potential concerns. Additionally, we plan to enrich Sec 1, 2.2, and 3.4, and the appendix, with more detailed motivation, background, and intuitive interpretations of the technologies.\n\n**Q3: Other Comments Or Suggestions**\n\nWe sincerely appreciate your suggestion and plan to reformat the figure to a vertical layout to enhance clarity.", "claims_and_evidence": "Claim 1: The author primarily discusses the limitations of Graph Neural Networks (GNNs) on undirected graphs, highlighting issues such as suboptimal data representations and inefficient learning at the model level.\n\nClaim 2: Additionally, the author argues that existing methods fail to fully capture the potential correlations between directed graph topology and node profiles at the data level. To address this, the proposed approach enhances existing methods by incorporating directed graph knowledge.\n\nEvidence 1: The author demonstrates that directed graphs provide richer and more diverse knowledge in terms of topology and node profiles, supporting this claim with case studies.\n\nEvidence 2: Extensive empirical studies are conducted in the experimental section to validate the proposed method.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "No, since I lack the background knowledge, I may not be able to check all the details of derivation.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "I still question whether such computationally expensive operations are truly necessary.", "essential_references_not_discussed": "I am not sure.", "other_strengths_and_weaknesses": "My main concern is the neccessary of such complex tree design and refine processes, and finally use the \"Monte Carlo\" method to enhance the efficiency.\n\n1. Could you give a real case that represents the so-called digraph data knowledge? Does it really exists?", "other_comments_or_suggestions": "1. The figure 2 is hard to distinguish the words and what you want to express, and also the Figure 1 (quite small)", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "No."}}, {"review_id": "bDUg361rtb", "reviewer": "Reviewer_tuGG", "summary": "This paper focuses on data-efficient representation learning for directed graphs and presents a novel online knowledge distillation framework based on a hierarchical tree structure. Leveraging this framework, the authors introduce EDEN, a new method that can be employed as a plug-and-play module to improve performance of existing directed graph neural networks or as an entirely new neural network architecture. The paper elucidates the novelty of the proposed knowledge distillation framework for graph learning and the effectiveness of the EDEN method through detailed textual presentations, charts, and relevant theoretical proofs, in a manner that is both reader-friendly and comprehensible. Moreover, extensive experiments have been conducted to validate the practicality of the proposed method.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1: Enhanced Explanations for Accessibility**\n\nWe appreciate your in-depth feedback and acknowledge that certain sections, particularly the formula interpretations, require background knowledge, which may pose challenges for readers. To improve readability, we will incorporate more intuitive explanations in the revision process, as suggested, ensuring greater accessibility for our audience.\n\nSpecifically, we plan to include the collection of annotations used for formulas in a tabular presentation to guide our readers in the main document. We will also read through the manuscript and add more intuitive explanations to display the functions and meaning of each formula to make them easy to comprehend. \n\nWe thank you again for your valuable comments to enhance the readability of our work. \n\n**W2: Detailed Discussion of Future Directions**\n\nWe sincerely appreciate your suggestions and acknowledge that some content may be brief due to our focus on presenting methodological details within the space constraints of the main document.\n\nWe will carry the obligation to provide more valuable insights for further development of this field, and we plan to provide additional information either within the main document or in the appendix, with clear references for accessibility for readers.\n\nSpecifically, we intend to open a new discussion in the appendix for potential proposals to further enhance the efficiency of EDEN with theoretical support and valuable related works for inspiring interested researchers. We also plan to add a clear indication in Sec 3.4 to guide our readers to this particular section in the appendix. Thank you! \n\n**W3: Thorough Review Needed**\n\nThank you for your notification. We will thoroughly review the manuscript to eliminate errors and ensure both accuracy and clarity of the content. More importantly, we will ensure that each formula is well interpreted for our readers with clearer annotations in the table for their convenience.", "claims_and_evidence": "This work explicitly highlights in Section 1: Introduction that existing directed graph neural network methods fail to fully leverage the rich knowledge embedded in data, thereby imposing a limited upper bound on performance. Building on this observation, the authors conduct an in-depth analysis of the issue. In Section 2: Preliminaries, they define knowledge from two dimensions—semantic and topological—using a hierarchical tree structure. This forms the basis for constructing a novel online knowledge distillation framework that paves the way for data-efficient graph learning. The evidence supporting the framework is meticulously presented and thoroughly substantiated in Section 3: Methodology, Section 4: Experiments, and Appendix.", "methods_and_evaluation_criteria": "For the Methods, the authors provide visual representations through Figure 1 and Figure 2, which facilitate reader comprehension. Additionally, they offer a granular exposition of the methods, relying on the formal problem definitions, equations, presentations, and theoretical proofs detailed in the Preliminaries and Methodology sections. Regarding the Evaluation Criteria, the authors elaborate on the experimental settings in the Experiments section, as demonstrated in Appendix A.10–A.13.", "theoretical_claims": "In this paper, the authors identify relevant theoretical foundations  and conduct a rigorous theoretical analysis. For instance, in the Preliminaries section, the concept of the hierarchical knowledge tree, inspired by hierarchical coding systems, is introduced. The related literature is comprehensively cited, and a clear problem definition is provided for ease of understanding. Furthermore, in the Methodology section, the authors perform a detailed theoretical analysis of the hierarchical online data knowledge distillation process from the perspective of neural mutual information estimation, thereby demonstrating its feasibility.", "experimental_designs_or_analyses": "The experimental designs and analyses in this work are both comprehensive and sound. Specifically, the performance evaluations across 14 datasets of varying scales and four downstream tasks demonstrate the superiority of this work. Building on this foundation, the ablation studies and sensitivity analyses are sufficiently thorough, providing robust evidence for the contributions of each module. Additionally, the authors leverage the appendix to offer supplementary convergence analyses, further enhancing the thoroughness and completeness of the experimental work.", "relation_to_broader_scientific_literature": "This work, taking directed graph representation learning as an example, proposes a universal data-efficient online knowledge distillation framework for graph learning. This is an inspiring attempt. Subsequent research in related fields can build upon the hierarchical knowledge tree defined by the authors to make more attempts and promote the vigorous development of data-centric graph learning.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "Strengths:\n1. Novel and Well-Motivated: The paper focuses on data-efficient graph learning, an issue of critical importance for the future development of the graph learning community. Building on this motivation, the authors introduce a novel framework for online data distillation using a hierarchical tree structure. This approach is highly intuitive and natural, leveraging the inherent properties of tree structures in a way that aligns well with human understanding. I believe this framework will be highly beneficial for future research in this area.\n\n2. Complete Methodology and Robust Theory: The authors provide a detailed exposition of the proposed online data distillation framework and the new method built upon it. The presentation is clear and easy to follow, making the complex concepts accessible to readers. The methodology is further enriched with clear definitions and thorough theoretical analysis, enhancing the interpretability and credibility of the proposed approach.\n3. Significant and Comprehensive Results: As a general method for data-efficient graph representation learning, the proposed EDEN method demonstrates substantial performance improvements when used as a plug-and-play module for existing methods. Additionally, it achieves state-of-the-art performance when employed as a standalone neural network architecture. The authors conduct extensive experiments across multiple datasets, downstream tasks, and backbone models, providing solid evidence of the method's effectiveness. The experimental results are robust and comprehensive, supporting the claims made in the paper.\n\n4. Well-Presented: The authors make excellent use of figures and clear presentations to simplify complex concepts, making the paper easy to understand. I appreciate the effort made by the authors to ensure that the content is well-organized and clearly communicated.\n\n\nWeaknesses:\n1. Enhanced Explanations for Accessibility: The authors provide detailed theoretical analysis in the methodology section, which is commendable. However, understanding these complex formulas may still be challenging for readers who are not well-versed in the field. Could the authors consider offering more intuitive descriptions or visual aids to lower the barrier to entry for a broader audience?\n3. Detailed Discussion of Future Directions: While the authors acknowledge the current limitations of their work, the discussion is somewhat brief. A more detailed exploration of promising future research directions would be valuable. Specifically, outlining potential avenues for achieving more data-efficient graph learning could provide valuable insights for the community.\n3. Thorough Review Needed: The authors are encouraged to conduct a comprehensive review of the entire manuscript to avoid potential typographical errors and minor mistakes. Ensuring the accuracy and clarity of the text will enhance the overall quality of the submission.", "other_comments_or_suggestions": "None.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PkmfnkKG8A", "reviewer": "Reviewer_zpYj", "summary": "This paper introduces ​entropy-driven digraph knowledge distillation (EDEN), a data-centric framework for representation learning on directed graphs. EDEN addresses the limitations of current directed Graph Neural Networks by leveraging ​directed structural measurements to construct a ​hierarchical knowledge tree (HKT), refined using ​mutual information of node features. This enables ​topology and profile knowledge transfer through a ​tree layer-wise distillation process. For downstream tasks like ​node classification and ​link prediction, EDEN employs a ​random-walk based technique. Experiments show that EDEN, as a hot-and-plug module, improves the performance of existing methods.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Due to the word limit imposed by the new regulations of ICML 2025 rebuttal, we have not provided detailed references, but we will gladly supply them in our subsequent discussions if needed.\n\n**Q1: Claims And Evidence**\n\nWe sincerely apologize for any concerns that may have arisen. We kindly refer you to our response to Reviewer RNnj Q1, which addresses similar issues. Based on this, we provide the following clarifications and outline our revision plans, hoping to further alleviate your concerns.\n\n(1) Important Directed Edges: Recent studies [LoG 2024, ICDE 2024, WWW 2025] have shown that considering edge directionality provides a novel approach to addressing the heterogeneity issue. These studies conducted extensive empirical analyses, highlighting the significant impact of extracting directed graph knowledge to enhance downstream task performance.\n\n(2) Our Contribution: Although they are effective, these methods vary in data knowledge extraction and lack a unified framework. Therefore, we propose EDEN, which aims to achieve more efficient data-centric graph learning and further enhance downstream task performance (i.e., fully explore the potential correlations between directed topology and node profiles at the data level). Based on this, inspired by your insightful comments, we have conducted additional experiments to further demonstrate the improvements of our framework on existing studies.\n\n| Models       |          | Empire    |           |          |   |      Rating     |           |          |\n|--------------|----------|-----------|-----------|----------|----------|-----------|-----------|----------|\n|              | Node-C   | Existence | Direction | Link-C   | Node-C   | Existence | Direction | Link-C   |\n| ADPA         | 79.3±0.4 | 67.3±0.5  | 54.2±0.4  | 59.0±0.5 | 44.8±0.5 | 77.8±0.4  | 83.7±0.3  | 64.5±0.5 |\n| ADPA+EDEN    | 81.7±0.4 | 68.6±0.4  | 55.3±0.6  | 60.7±0.3 | 46.6±0.3 | 79.4±0.5  | 85.2±0.2  | 66.0±0.6 |\n| MAP++        | 79.5±0.3 | 67.6±0.6  | 54.7±0.5  | 59.8±0.4 | 45.4±0.5 | 78.5±0.3  | 84.4±0.3  | 65.6±0.4 |\n| (MAP++)+EDEN | 81.4±0.6 | 69.0±0.5  | 56.4±0.4  | 61.2±0.4 | 47.5±0.4 | 79.8±0.3  | 85.6±0.4  | 66.8±0.5 |\n\nWe apologize if our initial presentation caused any misunderstandings. In the revised submission, we will reorganize the motivation, references, and experimental results based on the above clarification for a clearer presentation.\n\n**Q2: Methods And Evaluation Criteria**\n\nPlease allow us to offer the following explanations and revision plans, which we trust will effectively address your concerns and enhance your confidence in our manuscript.\n\n**Enhancing the Readability**\n\nWe plan to add intuitive explanations and background about formulas and introduce a table that includes mathematical symbol definitions. Taking Eq. (1–3) as examples, we will provide more background on the concept of Shannon entropy and graph mining.\n\n**Scalability Test and Performance**\n\nWe fully agree on the importance of enhancing EDEN's scalability. However, as the first data-centric digraph learning framework, some complexity is inevitably introduced. We kindly ask for your understanding in this regard. That being said, we introduce a lightweight alternative in Sec 3.4, which has been evaluated on the million-scale Arxiv and WikiTalk and renders it comparable to or even superior to the best baseline (Table 1 and Figure 3). We also plan to provide a more detailed discussion on potential efficiency optimizations, as elaborated in our responses to Reviewer RNnj Q1.\n\n**Q3: Experimental Designs Or Analyses**\n\nThanks for your valuable suggestions. We add ADPA (ICDE 2024) and MAP (WWW 2025) as additional baselines to the already existing Dir-GNN (LoG 2024) and HoloNet (ICLR 2024) as follows:\n\n| Node-level | CoraML   | CiteSeer | WikiCS   | Tolokers | Arxiv    |\n| ---------- | -------- | -------- | -------- | -------- | -------- |\n| ADPA       | 83.2±0.6 | 64.4±0.6 | 80.2±0.5 | 79.5±0.3 | 67.8±0.5 |\n| MAP++      | 83.5±0.4 | 64.7±0.7 | 79.8±0.4 | 80.1±0.3 | 68.2±0.4 |\n| EDEN       | 84.6±0.5 | 65.8±0.6 | 81.4±0.3 | 81.3±0.2 | 69.7±0.3 |\n\n| Link-level |                 | Slashdot       |              |                 | WikiTalk       |              |\n| ---------- | --------------- | -------------- | ------------ | --------------- | -------------- | ------------ |\n|            | Existence (AUC) | Direction (AP) | Link-C (ACC) | Existence (AUC) | Direction (AP) | Link-C (ACC) |\n| ADPA       | 90.9±0.1        | 92.6±0.1       | 86.0±0.1     | 94.8±0.2        | 90.8±0.1       | 90.6±0.1     |\n| MAP++      | 91.2±0.0        | 92.7±0.1       | 86.4±0.1     | 94.7±0.1        | 90.6±0.1       | 90.5±0.0     |\n| EDEN       | 91.8±0.1        | 93.1±0.0       | 87.1±0.2     | 95.4±0.1        | 91.7±0.1       | 91.0±0.1     |\n\n**Q4: Supplementary Material**\n\nThank you for your notification, we will include implementation details in the README to guide any interested researchers to implement our method.", "claims_and_evidence": "The motivation for the proposed method lacks sufficient clarity. The claim that existing methods \"fail to fully explore the potential correlations between directed topology and node profiles at the data level\" is not adequately substantiated. Illustrating how these limitations impact downstream tasks like node classification or link prediction would provide a clearer rationale for the necessity of the proposed method.", "methods_and_evaluation_criteria": "The paper suffers from a ​lack of necessary details, which hinder readability and comprehension. Many technical definitions, such as the ​structural measurements (Eqs 1-3), are presented without intuitive explanations. These shortcomings make the paper challenging to grasp and weaken its overall accessibility.\n\nTo comprehensively assess the ​scalability and efficiency of EDEN, it is crucial to include ​larger datasets in the evaluation, particularly to test its performance on ​large-scale graphs.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Are there any ​latest methodologies from 2024 that could serve as baselines, and if so, should they be included to ensure a more comprehensive and up-to-date evaluation?", "relation_to_broader_scientific_literature": "The topic is quite relevant.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "See all sections above.", "other_comments_or_suggestions": "See all sections above.", "questions_for_authors": "See \"Experimental Designs Or Analyses\"", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Fb05XoM9lA", "reviewer": "Reviewer_Yzad", "summary": "This paper proposes a general data-centric directed graph online knowledge distillation framework called EDEN. The framework achieves data-centric machine learning, guided by the proposed hierarchical encoding theory for the graph-structured data. The paper conducts experiments to validate the efficacy of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: Claims And Evidence**\n\nWe sincerely apologize for the insufficient explanation in our initial submission, which may have caused confusion. We kindly ask you to refer to our response to Reviewer RNnj Q2, where we provided an example of the hierarchical structure of directed graphs in the real world using citation networks. We also elaborated on our motivation for employing trees as a hierarchical data structure for data organization and learning, which is inspired by other relevant references.\n\nIf possible, we will include this real-world example of hierarchical structures in directed graphs in Sec 1 and the appendix of the revised submission, along with additional references, to provide a clearer presentation. \n\nWe hope that the above responses address your concerns and enhance your confidence in our manuscript.\n\n**Q2: Methods And Evaluation Criteria**\n\nWe sincerely appreciate your valuable insights, and please allow us to offer the following justifications: \n\n(1) Scalability Issues and Lightweight Alternative: Your concerns about scalability issues regarding our proposed EDEN are valid, and as a matter of fact, we have recognized it as the main bottleneck that requires our future efforts to optimize it further based on the lightweight alternative of EDEN in Sec 3.4. We kindly refer you to our response to Reviewer RNnj Q1 & Q2, which includes the theoretical motivation and implementation details of the current lightweight alternative, as well as the discussion on potential future efficiency improvements and the revision plans in the new submission.\n\n(2) Probably Missed Link-level Test: In response to your concerns regarding the absence of results for the link prediction task, we apologize for the lack of a direct indication in the main text, which may have led to your omission. In fact, we have already provided additional link-level evaluations for all datasets in Appendix A.14 of the initial submission. We fully intend to address this in the subsequent editing process.\n\nOnce again, we thank your valuable comment and hope our response can address your concern.\n\n**Q3: Other Strengths And Weaknesses**\n\nThank you for your comments. We acknowledge that the current abstract may be more extensive than typical abstracts. Our aim was to provide detailed explanations to help readers understand the intended task and the design of EDEN. However, we recognize the need for improvements and will refine the abstract during the editing process to enhance clarity and conciseness while ensuring intuitive interpretations of EDEN’s core contributions for future readers.", "claims_and_evidence": "It is not clear why (at least should give some examples or references) \"real-world digraphs commonly exhibit hierarchical structure\" (LHS of line 206, line 207).", "methods_and_evaluation_criteria": "The method seems valid but requires lightweight adaptation due to scalability issues. There are only two dataests for link prediction though, which may not be sufficient (but acceptable).", "theoretical_claims": "The claims seem correct, but I did not check carefully.", "experimental_designs_or_analyses": "The designs and analyses seem valid.", "relation_to_broader_scientific_literature": "There seems to be good relations to existing literature, and with novel contributions.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The paper has somewhat too long an abstract, and it is not friendly to non-experts in the specific field.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces EDEN, an entropy-driven approach for data-centric directed graph learning. The framework constructs a Hierarchical Knowledge Tree (HKT) to distill structural and semantic knowledge from directed graphs, serving as a plug-and-play module to enhance existing graph neural networks. EDEN shows strong performance improvements on various directed graph datasets and tasks.

Strengths:

-	Novelty and Clear Motivation: It introduces a novel online knowledge distillation framework for directed graphs, leveraging hierarchical knowledge trees and mutual information to enhance data efficiency.
-	Complete Methodology and Theory: The paper provides detailed theoretical analysis and clear definitions, making the approach interpretable and credible.
-	Comprehensive Results: Extensive experiments on diverse datasets and tasks show strong empirical improvements.

Weaknesses:

-	Complexity and Scalability Concerns: While a lightweight version exists, the full framework’s computational cost may limit large-scale applicability.
-	Readability and Clarity: Some technical sections lack intuitive explanations, making the method harder to follow.

The work is significant in the field of data-centric graph learning, offering a new perspective for directed graph learning. Key concerns of reviews included clarity of motivation, baseline comparisons, and scalability, which the authors addressed in rebuttal with additional experiments and explanations.
Decision: Accept (poster)