Title: HGDL: Heterogeneous Graph Label Distribution Learning
Venue: NeurIPS 2024
Accepted: yes
Keywords: Heterogeneous Graph

Graph Representation Learning

Label Distribution Learning
Abstract: Label Distribution Learning (LDL) has been extensively studied in IID data applications such as computer vision, thanks to its more generic setting over single-label and multi-label classification. 
This paper advances LDL into graph domains and aims to tackle a novel and fundamental
heterogeneous graph label distribution learning (HGDL) problem.
We argue that 
the graph heterogeneity reflected on node types, node attributes, and  neighborhood structures can 
impose significant challenges for generalizing 
LDL onto graphs. 
To address the challenges, we propose a new 
learning framework with two key components: 
1) proactive graph topology homogenization, 
and 2) topology and content consistency-aware graph transformer. 
Specifically, 
the former learns optimal information aggregation between meta-paths, so that the node
heterogeneity can be proactively addressed prior to the succeeding embedding learning; the latter leverages an attention mechanism to learn consistency between meta-path and node attributes, allowing network topology and nodal attributes to be equally emphasized during the label distribution learning. By using KL-divergence and additional constraints, \method~delivers 
an end-to-end solution for learning and predicting label distribution for nodes. 
Both theoretical and empirical studies substantiate 
the effectiveness of our HGDL approach.
Our code and datasets are available at https://github.com/Listener-Watcher/HGDL.

†

Abstract

Label Distribution Learning (LDL) has been extensively studied in IID data applications such as computer vision, thanks to its more generic setting over single-label and multi-label classification. This paper advances LDL into graph domains and aims to tackle a novel and fundamental heterogeneous graph label distribution learning ( HGDL ) problem. We argue that the graph heterogeneity reflected on node types, node attributes, and neighborhood structures can impose significant challenges for generalizing LDL onto graphs. To address the challenges, we propose a new learning framework with two key components: 1) proactive graph topology homogenization, and 2) topology and content consistency-aware graph transformer. Specifically, the former learns optimal information aggregation between meta-paths, so that the node heterogeneity can be proactively addressed prior to the succeeding embedding learning; the latter leverages an attention mechanism to learn consistency between meta-path and node attributes, allowing network topology and nodal attributes to be equally emphasized during the label distribution learning. By using KL-divergence and additional constraints, HGDL delivers an end-to-end solution for learning and predicting label distribution for nodes. Both theoretical and empirical studies substantiate the effectiveness of our HGDL approach. Our code and datasets are available at https://github.com/Listener-Watcher/HGDL .

1 Introduction

Definite supervision signals are often postulated in learning settings [3, 4]; yet, data generated from the real world tend to present inherent ambiguity, imposing challenges on assertive classifiers that predict instances into single or multiple classes. Label Distribution Learning (LDL) [5, 6, 7, 8, 9] has emerged to navigate label ambiguity by pursuing a mapping from instances to their class distributions. Each distribution quantifies the descriptive degrees of various classes given a specific instance.

However, the existing LDL studies mainly [10, 6, 7, 11] focus on independent and identically distributed (IID) data, such as images or texts, which do not generalize well on graphs . In fact, the topological structure underlying instances may provide invaluable information for label distribution learning. For example, in the task of urban planning, recent learning models have been employed to predict the point of interests (POIs) of local regions [12, 13, 14, 15]. LDL can further extend this task by providing the regional distributions over all POIs, which lends a finer-granular delineation of urban regional functionality instead of single- or multi-class classification. To wit, for a region that mixes four POIs (classes): housing, healthcare, education and worship, unlike other models assertively classify it into one or multiple POI(s), LDL model can provide insights of the functional degrees of all four POIs in this region, as shown in Figure 1. Nevertheless, existing LDL studies overlook the

urban topology, which can be rendered from, e.g., the taxi services across regions [2], missing out critical city traffic patterns that are highly correlated with regional functionalities. For instance, the regions with balanced POI distributions (e.g., R 2 and R 3 ) are less likely to form connections with other nodes compared to regions heavily skewed towards a single class (e.g., R 4 ), as their residents enjoy fewer needs to travel to other regions for services such as education and healthcare.

In this paper, we aim to enable and generalize the label distribution learning paradigm in networked data. Two technical challenges confront our study. First, real-world graphs are mostly heterogeneous, comprising diverse types of nodes for better expressiveness. Graph heterogeneity complicates the message-passing between nodes of a specific type (e.g., residence), as the label distributions of those nodes are influenced by their neighboring nodes that may vary in terms of types, content, and topological features. Simply leveraging node embeddings generated from message-passing for LDL will thus not work well [16, 17]. To aid, although meta-path aggregation [18] is seemingly viable, it necessitates extensive domain knowledge and expertise to craft meta-paths for each node type with respect to their label distributions; given the combinatorial number of possible meta-paths in large heterogeneous graphs, searching for the optimal meta-path for LDL is costly, laborious, and time-demanding.

Second, graph topology and nodal features may suggest inconsistent label distributions, where nodes sharing similar contents are positioned far apart on the graph topology. The inconsistency is furthered in heterogeneous graphs, where nodes of the same type often connect through other intermediary types, resulting in substantial topological distances between them. Unlike traditional LDL that focuses on instance vectors only, an effective LDL model on graphs require harmonizing nodal contents with topological structures for a unified representation. The impact of distantly positioned nodes within a graph is substantially diminished, consequently steering the LDL model to prioritize individual nodal vectors, leading to compromised

s

1

R3

R1

R 4

c

1

c

2

c

3

c

4

s

2

c

1

c

2

c

3

c

4

c

1

c

2

c

3

c

4

c

1

c

2

c

3

c

4

s

3

L 2

T 1

L 1

service

service

Leisure

Transit

Leisure

R2

service

c

1

Housing

c

2

Healthcare

c

3

Education

c

4

Worship

Classes (POIs)

residence

residence

residence

residence

?

?

?

?

node representations in which the informative patterns embedded in their neighborhood structures are overlooked. Such patterns, which may significantly enhance label distribution predictions as illustrated in Figure 1, are neglected, undermining the LDL model effectiveness.

To overcome the challenges, we propose a new learning framework, coined Heterogeneous Graph label Distribution Learning ( HGDL ). Specifically, to tame the graph heterogeneity, HGDL learns the optimal graph topology of the target nodes from multiple homogeneous structures searched with various meta-path schemes through a tailored attention mechanism. The node embeddings are then generated by harmonizing the nodal features and the learned meta-path graph using a transformer architecture. A joint optimization objective is crafted based on the distance between true and predicted label distributions of the target nodes from their resultant embeddings, which unifies the learning of meta-path graph topology and the feature-topology harmonization function in an end-to-end fashion.

A key innovation of HGDL is that it changes existing heterogeneous graph learning paradigm from reactive (meaning that aggregation of different meta-paths are done after embedding learning from individual meta-path), to be proactive (meaning that aggregation are done before embedding learning). Combined with attention and transformer mechanisms to adjust individual meta-paths' interplay, and align with nodal features, HGDL deliver significantly better performance over alternatives. Our theoretical analysis assures that HGDL outperforms that of using an arbitrary meta-path graph, and HGDL 's topology and feature consistency learning sparsifies network connectivity, intermediately encouraging tightens the error-bound, resulting in better model generalization.

Specific contributions of this paper are as follows:

This study pioneers the exploration of LDL problem in heterogeneous graphs. The learning problem enjoys practical implications such as for urban functionality delineation (presented in Sec 6.1) and, to our knowledge, has not yet been explored by any contemporary research.

We propose an end-to-end HGDL learning approach to jointly learn an optimal meta-path graph topology and align it with nodal features for consistent message-passing. Our approach is surprisingly simple yet effective, with its performance evidenced by theoretical underpinnings. Our approach and its analysis are presented in Sections 4 and 5, respectively.

Empirical study has been carried out over five graph datasets that span domains of biomedicine, scholarly network, business network, and urban planning. Experimental results substantiate the effectiveness of our approach over rival models, documented in Section 6.

2 Related Work

Label Distribution Learning (LDL) strives to learn a mapping from input to a distribution that profiles the descriptive degrees of classes associated with it [5, 10, 6, 7, 11]. Existing LDL methods fall into three categories, namely, problem transformation (PT), algorithm adaption (AA), and specialized algorithm (SA). PT methods transform LDL as multiple single-label learning tasks, using with label probabilities [19], and AA approaches revise mainstream learning algorithm to fit the LDL loss. SA algorithms are most commonly used because LDL learning is driven by new algorithm designs. Label correlation has been found to benefit the label distribution learning, where approaches were proposed to encode label correlation to a distance to measure the similarity of any two labels [6]. Later, low-rank approximation is used to construct label correlation matrix to capture the global label correlations [7] Instead of exploring common features for all labels, label-specific features [10] for each label are used to enhance the LDL model. Exploring feature-label and label-label correlation [9] has recently been studied in generalizable label distribution learning for cross domain learning. A Gaussian label distribution learning method [11] employs a parametric model underpinned by an optimization strategy assessing KL-divergence distance between Gaussian distributions, followed by a regression loss to normalize the KL-divergence distance. Noticing the difficulty to obtain ground-truth label distributions, Label Enhancement [20] is commonly used to recover label distributions from logical labels. Our research further push LDL to be generalized onto heterogeneous graphs, which have been overlooked by existing research. Although a recent study [21] explored using LDL in topological spaces, it focused on homogeneous graphs only and cannot work in the setting of more than one node type. Thus, the studied problem in [21] and its challenges differ from ours.

Heterogeneous Graph Neural Networks have drawn extensive attention in graph learning [16, 17, 22, 18, 23, 24], because the graph heterogeneity imposes considerable challenge to model the interplay among various node types, features, labels, and network topology. Using meta-path to aggregate information from different types of nodes/edges is a common approach for heterogeneous graph learning. HetGNN [17, 18] designs graph neural networks to encode features for each type of neighbors and then aggregates neighbors' representation with respect to different types. This provides a way for GNN to deal with heterogeneous graph structures and node attributes. HAN [25] introduces attention mechanisms to heterogeneous graph learning, where attentions are applied to embedding features learned from homogeneous networks, each created from a meta-path. By doing so, attentions serve as a weighting mechanism automatically determining the importance of each meta-path for learning. Using transformers for heterogeneous networks has also been investigated recently. For example. HGT [26] designs node- and edge-type dependent parameters to characterize the heterogeneous attention over each edge, allowing this method to learn representations for different types of nodes and edges. SeHGNN [27] proposes a transformer based semantic fusion module, allowing feature fusion from different meta-paths. Our research is fundamentally different from existing work in two aspects: 1) we study LDL learning for heterogeneous networks, and 2) we propose a new way to aggregate and align information for heterogeneous network.

3 Preliminaries

Notations. A heterogeneous graph is denoted by G = { V, E, X, Y } associated with a node type mapping ϕ : V ↦→ T v and an edge type mapping φ : E ↦→ T e , with T v and T e the predefined and finite sets of nodes and edges, respectively, and |T v | ≥ 2 . Denote t ι ∈ T v as the node type of our interest, and suppose in total n nodes are of this type. Without loss of generality, we have ϕ ( v 1 ) = . . . = ϕ ( v n ) = t ι , and V t ι = { v 1 , . . . , v n } ⊂ V . We deem these n nodes as our target nodes , using a feature matrix X ∈ R n × m to describe their nodal contents, where each node contains

. . .

P 1 : R-S-R

P k : R-L-T-L-R

meta-path

meta-path

~

. . .

l

reg=DKL(

[:,

i

] || U[1, k])

ϴ

[:,1] · LapNorm(A 1 )

ϴ

[:,k] · LapNorm(A k )

1

2

3

Sec 4.1

Sec 4.2

Sec 4.3

ZQ(ZK) T

R3

R2

R1

R4

R3

R2

R1

R4

Meta-Path

Attention

Scores

||

R1

R1

R3

R3

R2

R2

R4

R4

R1

R1

R3

R3

R2

R2

R4

R4

4

5

Target Node Vectors

Predicted Distributions Y

Feature Attention Adjacency

KL-divergence

Attention Regularizer

R1

R1

R3

R3

R2

R2

R4

R4

R1

R1

R3

R3

R2

R2

R4

R4

Ground Truth Y

̂

. . .

. . .

. . .

. . .

. . .

. . .

. . .

1

1

1

1

1

1

Z=GCN(A;X)

0.70.050.220.03

0.10.050.030.82

0.7

0.7

1

0.20.10.60.10

0.5

1

0.60.3

0

0

0.1

↑

0.20.30.40.1

GCN(A2;Z)

0.7

0.3

lKL=DKL(|Y)

0.10.20.60.1

Q

minimize(lkL-lre

1

R4

S3

R

W

2

C1C2C3C4

W

R

C1 C2 C3 C

1

1

an m -dimensional feature vector. A meta-path P is defined as a relational sequence in form of t 1 r 1 → t 2 . . . r i → t i r i +1 → t i +1 . . . → t l (abbreviated as P = ( t 1 t 2 . . . t i t i +1 . . . t l ), where ( t i t i +1 ) ∈ T e describes the composite relation between a pair of node types. By defining a meta-path P with same first and last node type as the target node type, i.e. t 1 = t l = t ι , we can use P to convert a heterogeneous network as a meta-path graph concerning only the target node type, which shall be discussed later in Section 4.1.

Problem Statement. In our HGDL problem, the goal is to learn a predictive mapping ℏ : ( G,X ) ↦→ Y , where Y ∈ [0 , 1] q is a distribution of descriptive labels over q classes. Let y i,j ∈ [0 , 1] be the probability that the node v i belongs to the j -th class, we have ∑ q j =1 y i,j = 1 . In this work, we follow a transductive learning regime [28] to allow the ground-truth label distributions known for a subset of target nodes V tr ⊂ V t ι during training. Our learned mapping ℏ is expected to generalize well so can make accurate prediction on the remaining target nodes V t ι \ V tr .

4 HGDL: The Proposed Approach

Overview. The proposed HGDL approach comprises three key components, as illustrated in Figure 2. First, for the target nodes belonging to the node type t ι of interest, HGDL generates multiple homogeneous meta-path graphs based on their original locations on the heterogeneous graph through meta-paths; the optimal graph topology of this node type is then learned from the homogeneous graphs via attention mechanism ( Sec 4.1 ). Second, HGDL learns the embeddings of the target nodes by harmonizing the information sourced from their feature space and the learned optimal topology using a transformer-like neural architecture ( Sec 4.2 ). Third, HGDL minimizes the distance between the predicted and ground-truth label distributions based on the learned node embeddings. We tailor an objective function to unify the three components into one end-to-end optimization problem, in which the optimal graph topology, the harmonization function of the feature and topological information, and the target node label distribution are jointly learned ( Sec 4.3 ).

4.1 Optimal Graph Topology Homogenization

For a heterogenous graph, by leveraging meta-path idea, multiple different meta-path homogeneous adjacency matrix can be obtained and they can be treated as multiple sources. Graph learning is about exchanging and updating information from neighbor nodes. A proper neighbor set is therefore important for a target node to learn correct distribution. Given multiple sources, each node will have

multiple neighbor sets to choose from for updating. Traditionally, embeddings are learned for all the neighbor sets, and then aggregation over embeddings is learned. Semantics over embeddings are hard to interpret and learn compared with directly learned from different neighbor sets.

To generate a meta-path graph from the original heterogeneous graph, interactions between the metapath and the heterogeneous graph path are used. Two nodes v i and v j are connected in the meta-path graph, if there exists a path connecting them in the heterogeneous graph, and the path follows the metapath. Given a meta path P = ( t 1 . . . t i t i +1 . . . t l ) , we say that a path p = ( v 1 , . . . , v i , v i +1 , . . . , v l ) in graph G follows the meta-path P , if ∀ i, ϕ ( v i ) = t i . Take graph in Fig. 1 as an example. Given meta-path P = ( r s r ) , which defines node type t 1 = r, t 2 = s, t 3 = r . Path p = ( r 2 , s 1 , r 3 ) in the heterogeneous graph follows P because all nodes in the path p satisfy ϕ ( r 2 ) = t 1 = r, ϕ ( s 1 ) = t 2 = s, ϕ ( r 3 ) = t 3 = r . Because path p = ( r 2 , s 1 , r 3 ) follows the meta-path P , an edge is used to connect r 2 and r 3 in the homogeneous meta-path graph constructed from P . Indeed, each meta-path defines a specific way of information propagation in a heterogeneous network, with resulted meta-path graph capturing unique relationships between target nodes. While defining a single meta-path is relatively easy, there often exists many meta-paths; aggregating a variety of meta-path graphs to support the downstream learning task is non-trivial.

After searching the meta-paths connecting the nodes of target type t ι , we generate a set of graphs A = { A 1 , . . . A k } , in which each adjacency A i ∈ { 0 , 1 } n × n captures the topological structure of the i -th meta-path-based homogeneous graph. Denoted by A i [ p, q ] = 1 means that two target nodes v p and v q , with ϕ ( v p ) = ϕ ( v q ) = t ι , are connected by a meta-path; otherwise, A i [ p, q ] = 0 . Unlike existing studies [25] that yield target node embeddings through reactive meta-path aggregation, where they aggregate local neighborhood information for each A i ∈ A to capture k separate meta-path topologies, our HGDL learns the optimal graph topology from A in a proactive fashion. Intuitively, HGDL learns node-level attention scores for various homogeneous graphs A i , to respect the fact that the neighboring nodes may pass messages with varying importance levels in local neighborhoods, while the meta-paths walking across nodes of types other than the target t ι . Revisit the motivating example demonstrated in Fig 1 where the residence nodes are deemed as the target, the meta-path linking through the service nodes dominates, as the residence nodes are more likely to be linked through service nodes instead of Transit and Leisure nodes. Specifically, the attention scores for the nodes in every A i ∈ A are learned in a GAT regime [29], defined as:

̸

where LapNorm( A i ) = D -1 2 i ( A i + I ) D -1 2 i ∈ R n × n denotes Laplacian normalization of A i , with D i being A i 's degree matrix and I is an identity matrix. This term mitigates the imbalanced degree distribution of the meta-path graphs. Denoted by W i 0 and W ′ 0 are the learnable GAT parameters, where W i 0 ∈ R n × f maps the meta-path topology of A i onto an f -dimensional semantic space. The operator ∥ A i ∈A {·} concatenates all k resultant node embedding matrices from meta-path graphs A , thereby producing an R n × k · f lookup matrix, where each node is associated with a k · f -dimensional embedding representation, and each latent f -dimension captures the local neighborhood structure of this node. Then, W ′ 0 ∈ R k · f × k summarize the f -dimensional latent space into one coefficient through convex combination, resulting in attention logits, which are fed into softmax ( · ) = exp( · ) / ∑ i exp( · ) to yield the attention matrix Θ = [0 , 1] n × k . Take the i -th column vector of Θ , denoted by Θ[: , i ] ∈ [0 , 1] n , we have the attention scores of n target nodes for message-passing in the i -th meta-path graph A i . We thus can deem ˜ A in Eq. (1) as the learned optimal graph topology, which is an element-wise linear combination of k meta-path topologies with the attention scores broadcast onto all n nodes. Note, ˜ A is asymmetric, namely ˜ A [ p, q ] = ˜ A [ q, p ] , ∀ p = q . This is because that the attention score of information aggregation from node v p to v q may be different from that from v q to v p , as their respective local neighborhood topologies naturally differ.

̸

4.2 Local Topology and Global Feature Consistency-Aware Graph Transformer

After obtaining the optimal topology ˜ A from all meta-path graphs A i , the next question is how to harmonize it with the feature information to better the target node embeddings. The benefit of such harmonization is evident. Revisiting the urban network in Fig 1, we can envision that a pair of residence nodes tend to be associated with similar embedding vectors because they enjoy two types

of consistencies: i) local neighborhood topology : their residents tend to travel to similar functional regions for leisure or service purposes, and ii) global feature space : they share similar contents such as house types and number of residing families. These local and global consistencies complement with each other, as the residence nodes having similar contents can be topologically faraway from each other on the urban network, and vice versa.

To harmonize the local and global consistencies, we are inspired by the recent graph transformers [30] and observe that the feature attention suggests a global adjacency matrix, which can be incorporated into the message-passing process. We define the graph transformer block as follows.

where Z ∈ R n × h is the node embeddings learned from the optimal graph topology ˜ A through local information aggregation, parameterized by W 1 ∈ R m × h . Denoted by ˜ A 2 ∈ [0 , 1] n × n is the normalized feature attention adjacency, where Q and K ∈ R h × h map the embedding matrix Z onto the latent query and key spaces, respectively, such that ( ZQ )( ZK ) ⊤ calculates an n × n node-level attention matrix with respect to the feature space information. Instead of normalizing the attention score by the hidden dimension h , we penalize the feature attention adjacency through an element-wise production ⊙ with the optimal meta-path topology ˜ A . The intuition behind Eq. (2) is that, for each target node, it aggregates information from those neighboring nodes only if their meta-path topology and feature space are both with high attention scores. In addition, Eq. (2) functions similarly to the edge dropout [31]; in lieu of randomly removing edges, we enforce a neighbor-set intersection, where the information is only propagated from the neighbors on which the feature space and meta-path topology both agree. Such an intersection sparsity thus lowers the degree of the resultant attention adjacency, thereby uplifting the learning efficacy, which will be substantiated later in Sec 5. Finally, denoted by H = LeakyReLU ( ˜ A 2 ZW 2 ) ∈ R n × h are the resultant node embeddings, capturing both local topology and global feature consistencies, which is parameterized by weight W 2 ∈ R h × h .

4.3 An End-to-End HGDL Objective Function

Based on the resultant target node embeddings H , we can predict their label distributions as ˆ Y = softmax ( ˜ A 2 HW 3 ) ∈ [0 , 1] n × q , where ˆ Y i = { ˆ y i,j } q j =1 is the predicted label distribution of node v i , among which ˆ y i,j denotes its predicted probability of belonging to the class j . The unified objective of our HGDL framework is defined as follows.

where the KL-divergence D KL ( Y ∥ ˆ Y ) gauges the discrepancy between the predicted and groundtruth label distributions of the target nodes [32]. The regularization term Ω gauges the distance between the attention scores of the i -th node across k meta-path typologies (denoted by Θ[ i, :] ) and a uniform distribution U [1 , k ] . We note the minus sign before Ω , thus minimizing this term encourages a larger KL-distance, thereby avoiding the trivial uniform attention distribution (meaning that for each node, the learned attention weights from different meta-paths are encouraged to be as different as possible). γ is a tuned parameter to balance the two terms.

5 Analysis

We follow the PAC-Bayes regime to analyze the theoretical performance of our HGDL algorithm by deriving its generalization error bound. We proceed analysis based on the meta-path graph adjacency matrices A = { A 1 , . . . A k } , which are searched from the heterogeneous graph G . Throughout the analysis, we assume the nodal feature representations to be residing in an ℓ 2 -ball of radius B . We argue this a mild assumption, because in implementation we can leverage the batch-norm layers to normalize the resultant node embeddings, such that ∥ h j i ∥ 2 ≤ B , where h j i denotes the i -th node's embedding resulted from the j -th hidden layer.

Let L G ( ℏ ) and L ( X, ˜ A ) ( ℏ ) denote the generalization risk over a graph distribution G and the empirical risk on the target node samples and the learned meta-path topology ( X, ˜ A ) , respectively, where ( X, ˜ A ) ∈ G iid ∼ G . We can define:

where ℓ ( · , · ) is a convex distance metric between two distributions that follows | ℓ ( u, p ) -ℓ ( u, q ) | ≤ ( √ p +1) ∥ p -q ∥ 2 , ∀ u, p, q ∈ R m . Denoted by ℏ ( X, ˜ A )[ i ] ∈ R q and y i ∈ [0 , 1] q the predicted and ground-truth label distribution of the i -th target node, respectively. Implementing KL-divergence, we have ℓ ( ℏ ( X, ˜ A )[ i ] , y i ) = ∑ q j =1 ℏ ( X, ˜ A )[ i, j ] ln( ℏ ( X, ˜ A )[ i, j ] /y i,j ) , where the predicted probability that node i belongs to the j -th class is denoted by ℏ ( X, ˜ A )[ i, j ] . By analyzing the performance of the learned meta-path graph topology ˜ A , we find that:

Theorem 1. Let E [ L ( X,A i ) ( ℏ )] be the empirical risk of using the i -th meta-path graph A i ∈ A for label distribution prediction. With the SGD step-size η , we have

Remark 1. Theorem 1 indicates that the empirical risk of HGDL is no larger than the minimum empirical risk incurred by training label distribution learner on the optimal meta-path graph, as the error bound on the RHS reduces to O (1 /n ) with constant k and η . With Stochastic Gradient Descent (SGD) optimizer, larger number of target nodes n will lead to more training updates over them, diminishing the O (1 /n ) bound faster. This finding substantiates the tightness of our meta-path learning strategy for the optimal graph topology ˜ A .

Due to page limits, we defer the proof of Theorem 1 and the rest analysis to the Supplement. We then analyze the generalization error bound of HGDL and find that:

Theorem 2. Let ℏ ∈ H : X ×G → R q be an l -layer message-passing neural network with maximum hidden dimension k , of which the i -th layer is parameterized by W i . Then for any δ, γ, B > 0 and l > 1 , with probability at least 1 -δ we have

where D ( W i ) = ∏ l i =1 ∥ W i ∥ 2 2 · ∑ l i =1 ( ∥ W i ∥ 2 F / ∥ W i ∥ 2 2 ) bounds the hypothesis space and b is a constant.

Remark 2. We remark several key observations from Theorems 1 and 2. First, the generalization capability of the algorithm is negatively impacted by a higher dimensional label space q . Second, the robustness of HGDL decreases with larger B values, which gauges the magnitude of perturbations thus the inherent high data variance. Third, as the graph neural network architecture becomes deeper (larger l ) or wider (larger k ), the generalization risk increases, suggesting the potential risk of model overfitting. Forth, with larger n , the generalization error bound diminishes, which indicates that the meta-path topology can be better delineated with an increased number of target nodes on the graph.

Remark 3. By combining Theorems 1 and 2, we observe that the generalization error bound of HGDL using ˜ A outperforms that of using an arbitrary meta-path graph A i . Further, it is easy to verify that the maximum degree of ˜ A , denoted by d , is smaller than that of A i , denoted by d i , i.e., d ≤ d i . ∀ i ∈ [ k ] . This rationalizes our graph transformer design in Sec 4.2, where the enforced topology and feature consistency in Eq. (2) sparsifies network connectivity, thereby intermediately encourages better model generalization.

6 Experiments

6.1 Experiment Setup

Benchmark Datasets To our best

knowledge, no heterogeneous graph dataset with ground-true label distributions currently exists. To level the comparison study, we prepare five datasets with ground-truth node label distributions using existing heterogeneous graphs, including DBLP [33], ACM [33], YELP, DRUG [34], and URBAN [1]. Table 1 summarizes the data statistics. A detailed description on the dataset creation and preprocessing, as well as their domain and label semantic meanings, has been deferred to the Supplement B due to space limitation.

Compared Models In total six competitors are identified for comparative study. As no model directly resolving the HGDL problem exists, we employ the state-of-the-art heterogeneous graph neural networks and integrate them KL-divergence loss to learn label distributions of nodes. They include: 1) GCNKL: A baseline that uses graph constructed from each meta-path to train a vanilla GCN [35], using KL-divergence as loss function, and reports the best meta-path result; 2) HANKL: This baseline uses HAN [25] to integrate embedding from different meta-paths; and 3) SeHGNNKL: This baseline uses SeHGNN [27], a transformer based approach, to aggregate meta-paths embedding with KLdivergence loss function. For ablation study, we further include three variants reduced from our proposed HGDL method, which include: 4) HGDL ¬ TH: it removes HGDL 's topology homogenization (Sec 4.1), which learns embedding from each meta-path graph and reports the best meta-path result; 5) HGDL ¬ transformer: it uses GCN instead of the transformer (Sec 4.2) to learn embedding to validate HGDL 's transformer for embedding learning; and 6) HGDLED: it replaces HGDL 's topology and feature consistence-aware graph transformer (Sec. 4.2) by using a random edge dropout method [31].

Evaluation Metrics To measure the discrepancy between two distributions, i.e., the predicted and true label distributions of target nodes, we identify six metrics: Cosine Distance (COD), Canberra Distance (CAD), Chebyshev Distance (CHD), Clark Distance (CLD); Intersection Score (IND), and Kullback-Leibler Divergence (KL). Their definitions and calculations are deferred to Supplement B.

(a) DRUG

(b) ACM

(c) DBLP

(d) YELP

(e) URBAN

12

1.0

0.2

edsde = Ed

IND

KL

HGDL

COD

CAD

CHD

CLD

IND

KL

p1 =apcpa

HGDL

2.00

1.75

1.50

Measure

0.75

0.50

0.25

0.00

COD

CAD

CHD

CLD

IND

KL

p1 = rsr

=d

= d

HGDL

COD

CAD

CHD

CLD

IND

KL

COD

CAD

pdp = td

P = dsd

pd5dp = Ed

HGDL

P = apa

2.00

1.75

1.50

Measure

1.00

0.75

0.50

00'0

COD

CAD

CHD

CLD

IND

KL

nuqun =td

p2=utbtu

HGDL

2.3

2.0

Measure

1.5

1.0

0.5

9.9

COD

CAD

CHD

CLD

IND

KL

p1 = apa

p = apspa

HGDL

and ClarkDistance)

ClarkDistance

KLDivergence

0.4

region_lower_bound

region_upper_bound

0.3

(KL

0.2

LossFunction

0.1

0.0

0.00

0.02

0.04

0.06

0.08

0.10

SingleProbabilityValue

6.2 Results

Table 2 summarizes results of all methods. Overall, our HGDL wins in 99 out of 180 settings, among which on average 20 out of 30 settings excel in COD, CHD, IND, and KL metrics, 11 out of 30 in CAD, and 8 out of 30 in CLD. On DURG, ACM, DBLP, and URBAN datasets, HGDL outperforms its competitors in 83% settings in COD, CHD, IND, and KL metrics 46% in CAD, and 20% in CLD. Beyond its overall better comparative performance, we make the following observation on HGDL . First, HANKL and SeHGNNKL achieve better performance on YELP and DBLP dataset for CAD and CLD metrics, but not on the other datasets and metrics. This shows that existing meta-path based methods cannot learn distribution prediction well. In general, these models show similar performance on YELP dataset. We hypothesize that this is due to the lack of rich feature information on YELP, of which the dimension of nodal features is 19 which is minimum across all datasets. Second, HGDL achieves the best results in KL-divergence by a large margin across all settings. On average, HGDL have a 15% improvement compared with the second best result across all datasets in KL-divergence. Given that KL-divergence is the loss objective in our framework, we extrapolate that HGDL converges well in terms of minimizing the distribution distance. Same observation can

be drawn from the validation loss curve as shown in Supplement C Figures 5, 6, and 7. In addition, on metrics being strongly related to KL divergence including COD, CHD, and IND, our HGDL also enjoys significant performance improvement over other models. Among the metrics, CLD metrics shows a different patterns in terms of KL divergence, we show in Figure 4 that CLD and KL has a tradeoff region in small probability distribution and therefore caused such difference.

Third, the ablation study between HGDL and its variants, i.e., HGDL ¬ transformer, HGDL ¬ TH and HGDLED, demonstrate clear benefits of topology homogenization and consistency-aware graph transformer in aggregating meta-paths and nodal features for LDL learning for heterogeneous graphs (more results are deferred to the Section E.1 in Supplement C; there, we observe that HGDLed has no improvement in KL-divergence with different edge drop rates compared to HGDL ¬ transformer, which is the model with 0 edge drop rate). We observe in Table 2 that HGDL ¬ transformer shows comparable performance on ACM and YELP by tying HGDL in five and six metrics, respectively; however, HGDL outperforms it in all settings in other three datasets. Likewise, HGDL ¬ TH ties HGDL across all metrics in YELP but is inferior to HGDL in all settings in other four datasets. HGDLED ties HGDL in six and five settings on YELP and URBAN, respectively, but is outperformed by HGDL for all other three datasets in all settings. The robust performance of HGDL can be attributed to two aspects. On the one hand, the improved results over those ablation variants suggest that our devised model components for proactive meta-path learning and attention modeling are indispensable. On other other hand, it substantiates the usefulness of our design that lets HGDL learn semantic fusion before the embedding learning. This end-to-end learning design provides a larger search space for embedding learning to find optimal solutions, whereas other methods that learn embedding and perform fusion in two independent stages may result in suboptimal node embeddings thus inferior LDL performance.

Fourth, even though the optimal meta-path choice may vary across different metrics and datasets, our HGDL that proactively learns to aggregate multiple meta-path graphs leads to the best performance in most cases. Figure 3 illustrates the performance from single meta-path graph and we can observe that our method outperforms the single best path results in all five datasets, with a larger improvement when the meta-path results are close (indicating each meta-path has similar information, e.g. , ACM dataset in Figure 3 (b)) and a smaller improvement when one meta-path is significantly inferior to others ( e.g. , DBLP dataset Figure 3 (c) where p 1 outperforms p 2 with a large margin). These results validate the tightness of Theorem 1 by demonstrating the optimality of the learned meta-path graph in our HGDL method.

6.3 Scalability Analysis

Denote the total number of nodes, hidden dimension size, and number of meta-path by n , f , and k , respectively. The number of learnable parameters is O ( n ) for graph topology homogenization, because HGDL requires learning an adjacency matrix from all meta-path, which involves knf + k 2 f training parameters ( i.e. O ( n ) complexity). Inducing adjacency matrix from features, i.e. the 2nd stage, only requires O (1) number of learnable parameters, same as vanilla GCN. As a result, HGDL has O ( n ) complexity. The runtime performance is detailed in Appendix H.3.

7 Conclusion

This paper explored a novel graph learning setting, namely, heterogeneous graph label distribution learning. Our goal is to predict label distributions of target nodes in a heterogeneous graph, which enables a finer-granular delineation of node properties compared to traditional single- or multiclass node classification. We demonstrated that the topological heterogeneity and inconsistency impose unique challenge for generalizing LDL into networked data, and proposed HGDL to overcome them. Specifically, HGDL proactively aggregates meta-paths to achieve optimal graph topology homogenization through attention mechanism, followed by a transformer-based approach to ensure topology and feature consistency for learning node label distributions. We analyzed the PAC-Bayes error bound of HGDL , and the result suggests the superiority of our design over those models learned from a single meta-path graph. Empirical results on five benchmark datasets validated the tightness of our analysis and substantiate that HGDL significantly outperformed its competitors.

Acknowledgment

This work has been supported in part by the National Science Foundation (NSF) under Grant Nos. IIS-2236578, IIS-2236579, IIS-2302786, IIS-2441449, IOS-2430224, and IOS-2446522.

References

N. L. Houssou, J.-l. Guillaume, and A. Prigent, 'A graph based approach for functional urban areas delineation,' in Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing , pp. 652-658, 2019.

A. Rossi, G. Barlacchi, M. Bianchini, and B. Lepri, 'Modelling taxi drivers' behaviour for the next destination prediction,' IEEE Transactions on Intelligent Transportation Systems , vol. 21, no. 7, pp. 2980-2989, 2019.

P. Kar and P. Jain, 'Supervised learning with similarity functions,' NeurIPS , vol. 25, 2012.

A. Hefny, C. Downey, and G. J. Gordon, 'Supervised learning for dynamical system learning,' NeurIPS , vol. 28, 2015.

X. Geng, 'Label distribution learning,' IEEE Trans. Knowl. Data Eng. , vol. 28, no. 7, pp. 17341748, 2016.

X. Jia, W. Li, J. Liu, et al. , 'Label distribution learning by exploiting label correlations,' in AAAI , 2018.

T. Ren, X. Jia, W. Li, and S. Zhao, 'Label distribution learning with label correlations via low-rank approximation,' in IJCAI , p. 3325-3331, 2019.

J. Wang and X. Geng, 'Theoretical analysis of label distribution learning,' in AAAI , vol. 33, pp. 5256-5263, 2019.

X. Zhao, L. Qi, Y. An, and X. Geng, 'Generalizable label distribution learning,' in Proceedings of the 31st ACM International Conference on Multimedia (MM-23) , p. 8932-8941, 2023.

T. Ren, X. Jia, W. Li, L. Chen, and Z. Li, 'Label distribution learning with label-specific features,' in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19) , pp. 3318-3324, International Joint Conferences on Artificial Intelligence Organization, 2019.

H. Xu, X. Liu, Q. Zhao, Y. Ma, C. Yan, and F. Dai, 'Gaussian label distribution learning for spherical image object detection,' in CVPR , 2023.

Z. Yao, Y. Fu, B. Liu, W. Hu, and H. Xiong, 'Representing urban functions through zone embedding with human mobility patterns,' in IJCAI , 2018.

Y. Luo, F.-l. Chung, and K. Chen, 'Urban region profiling via multi-graph representation learning,' in CIKM , pp. 4294-4298, 2022.

Y. Zheng, Y. Lin, L. Zhao, T. Wu, D. Jin, and Y. Li, 'Spatial planning of urban communities via deep reinforcement learning,' Nature Computational Science , vol. 3, no. 9, pp. 748-762, 2023.

Y. Liu, J. Ding, Y. Fu, and Y. Li, 'Urbankg: An urban knowledge graph system,' ACM Transactions on Intelligent Systems and Technology , vol. 14, no. 4, pp. 1-25, 2023.

Q. Lv, M. Ding, Q. Liu, Y. Chen, W. Feng, S. He, C. Zhou, J. Jiang, Y. Dong, and J. Tang, 'Are we really making much progress? revisiting, benchmarking and refining heterogeneous graph neural networks,' in SIGKDD , pp. 1150-1160, 2021.

C. Zhang, D. Song, C. Huang, A. Swami, and N. Chawla, 'Heterogeneous graph neural network,' in Proc. of KDD , pp. 793-803, 2019.

X. Fu, J. Zhang, Z. Meng, and I. King, 'Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding,' in WWW , pp. 2331-2341, 2020.

H. Borchani, G. Varando, C. Bielza, and P. Larranaga, 'A survey on multi-output regression,' Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , vol. 5, 07 2015.

Y. Wang, Y. Zhou, J. Zhu, X. Liu, W. Yan, and Z. Tian, 'Contrastive label enhancement,' in Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI-23) , pp. 4353-4361, 2023.

Y. Jin, R. Gao, Y. He, and X. Zhu, 'Gldl: Graph label distribution learning,' in Proceedings of the 38th Annual AAAI Conference on Artificial Intelligence , 2024.

Y. Dong, Z. Hu, K. Wang, Y. Sun, and J. Tang, 'Heterogeneous network representation learning,' in Proc. of the 29th International Joint Conf. on Artificial Intelligence (IJCAI-20) , pp. 48614867, 7 2020.

Y. Jing, Y . Yang, X. Wang, M. Song, and D. Tao, 'Amalgamating knowledge from heterogeneous graph neural networks,' in CVPR , pp. 15709-15718, 2021.

X. Wang, N. Liu, H. Han, and C. Shi, 'Self-supervised heterogeneous graph neural network with co-contrastive learning,' in SIGKDD , pp. 1726-1736, 2021.

W. Xiao, J. Houye, S. Chuan, W. Bai, C. Peng, Y. P., and Y. Yanfang, 'Heterogeneous graph attention network,' WWW , 2019.

Z. Hu, Y. Dong, K. Wang, and Y. Sun, 'Heterogeneous graph transformer,' in Proceedings of The Web Conference , p. 2704-2710, 2020.

X. Yang, M. Yan, S. Pan, X. Ye, and D. Fan, 'Simple and efficient heterogeneous graph neural network,' Proceedings of the AAAI Conference on Artificial Intelligence , pp. 10816-10824, 2023.

M. Wan, Y. Ouyang, L. Kaplan, and J. Han, 'Graph regularized meta-path based transductive regression in heterogeneous information network,' in SDM , pp. 918-926, 2015.

P. Veliˇ ckovi´ c, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, 'Graph attention networks,' in ICLR , 2018.

V. P. Dwivedi and X. Bresson, 'A generalization of transformer networks to graphs,' 2021.

Y. Rong, W. Huang, T. Xu, and J. Huang, 'Dropedge: Towards deep graph convolutional networks on node classification,' in International Conference on Learning Representations (ICLR) , 2020.

S. Kullback and R. A. Leibler, 'On information and sufficiency,' The annals of mathematical statistics , vol. 22, no. 1, pp. 79-86, 1951.

J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su, 'Arnetminer: Extraction and mining of academic social networks,' in KDD , pp. 990-998, 2008.

Y. Gu, S. Zheng, Q. Yin, R. Jiang, and J. Li, 'REDDA: Integrating multiple biological relations to heterogeneous graph neural network for drug-disease association prediction,' Computers in Biology and Medicine , 11 2022.

T. N. Kipf and M. Welling, 'Semi-supervised classification with graph convolutional networks,' in ICLR , 2017.

G. Landrum, P. Tosco, B. Kelley, Ric, D. Cosgrove, sriniker, gedeck, R. Vianello, NadineSchneider, E. Kawashima, G. Jones, D. N, A. Dalke, B. Cole, M. Swain, S. Turk, AlexanderSavelyev, A. Vaucher, M. Wójcikowski, I. Take, V. F. Scalfani, D. Probst, K. Ujihara, guillaume godin, A. Pahl, R. Walker, J. Lehtivarjo, F. Berenger, jasondbiggs, and strets123, 'rdkit/rdkit: 2023_09_4 (q3 2023) release,' Jan. 2024.

D. S. Himmelstein, 'User-friendly extensions to mesh v1.0,' Feb. 2016.

V. D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre, 'Fast unfolding of communities in large networks,' Journal of statistical mechanics: theory and experiment , vol. 2008, no. 10, p. P10008, 2008.

Q. Mao, Z. Liu, C. Liu, and J. Sun, 'Hinormer: Representation learning on heterogeneous information networks with graph transformer,' 2023.

N. Cesa-Bianchi and G. Lugosi, Prediction, learning, and games . Cambridge university press, 2006.

P. L. Bartlett, D. J. Foster, and M. J. Telgarsky, 'Spectrally-normalized margin bounds for neural networks,' NeurIPS , vol. 30, 2017.

B. Neyshabur, S. Bhojanapalli, and N. Srebro, 'A pac-bayesian approach to spectrallynormalized margin bounds for neural networks,' in ICLR , 2018.

R. Liao, R. Urtasun, and R. Zemel, 'A pac-bayesian approach to generalization bounds for graph neural networks,' in ICLR , 2020.

J. A. Tropp, 'User-friendly tail bounds for sums of random matrices,' Foundations of computational mathematics , vol. 12, pp. 389-434, 2012.

P. L. Bartlett and S. Mendelson, 'Rademacher and gaussian complexities: Risk bounds and structural results,' Journal of Machine Learning Research , vol. 3, no. Nov, pp. 463-482, 2002.

R. E. Schapire and Y. Freund, Foundations of machine learning , pp. 23-52. Mit Press, 2012.

A. Maurer, 'A vector-contraction inequality for rademacher complexities,' in ALT , pp. 3-17, Springer, 2016.

S. M. Kakade, K. Sridharan, and A. Tewari, 'On the complexity of linear prediction: Risk bounds, margin bounds, and regularization,' NeurIPS , vol. 21, 2008.

S.-H. Cha, 'Comprehensive survey on distance/similarity measures between probability density functions,' City , vol. 1, no. 2, p. 1, 2007.

A Appendix / supplemental material

B Roadmap

A structured guide to navigate the supplement content is organized as follows.

Supplement A: Supplement A presents the pseudo-code of our proposed HGDL algorithm.

Supplement B: Supplement B reports datasets and baseline methods used for the experimental study.

Supplement C: Supplement C reports experimental settings and additional results.

Supplement D: Supplement D elaborates the proof of Theorem 1 as introduced in Section 5 of the main manuscript.

Supplement E: Supplement E elaborates the proof of Theorem 2 as introduced in Section 5 of the main manuscript.

Supplement F: Supplement F carries out additional theoretical analysis explaining the rationality of HGDL for label distribution learning.

C Supplement A: Pseudo-code and Implementation Details

Algorithm 1 lists the main steps of the HGDL framework where Lines 1 to 6 generate meta-path graphs and their adjacency matrices. Lines 7 to 10 denote the optimal graph topology homogenization (Sec 4.1 of the main manuscript), and Lines 11 to 15 are local topology and global feature consistencyaware graph transformer (Sec 4.2 of the main manuscript). Lines 16 to 20 denote the loss terms and the model training process.

D Supplement B: Experiments and Datasets

D.1 Dataset Description

Because no benchmark heterogeneous graph datasets with label distributions currently exist, we create four datasets using existing heterogeneous graphs including DBLP [33], ACM [33], YELP, and DRUG dataset [34]. Table 3 reports number of nodes/edges, number of each node/edge types, and number of labels for each dataset. Table 4 reports meta-paths and label semantic for each datasets. Number of features are refer to the content attribute/features of the target node types. In our experiments, we did not use any content information of non-target nodes (so only target nodes have features).

DBLP is a citation network [33] with four types of nodes (author, paper, conference, and term nodes). Author nodes are our target nodes. To construct a label distribution, we labeled each author with a distribution of conference areas, using the history of each author's published papers at different conference areas. The author's node features are a bag of words of all papers published by the author.

ACM is a citation network from [33] with five types of nodes (author, paper, conference, affiliation, and subject nodes). Author nodes are our target nodes. We first extract one third of the authors and then labeled them based on the subjects of their published papers. Similar to the DBLP dataset, each author's features are also a bag-of-word representation of the author's papers.

YELP is an open-access review network with four types of nodes (business, user, tip, and review nodes). User nodes are our target nodes. We label each user by counting the ratings of the business each user made reviews on and normalize the countings of each rate to a distribution. User's features are the bag of words of their reviews.

DRUG is a drug-disease-protein-gene network from [34], with four types of nodes (drug, disease, protein, and gene nodes). Drug nodes are our target nodes. The label distribution of each drug node is obtained by using associated disease etiology (using top-level categories from Medical Subject Headings, MeSH, disease database). The node features is

Class 1

Class 2

Class 3

Class 4

Class 5

Class 6

Class 7

Class 8

Class 9

Class 10

Class 11

Class 12

Class 13

Class 14

0.8363

±

0.2439

0.0049

±

0.0328

0.0045

±

0.0325

0.0067

±

0.0389

0.0112

±

0.0516

0.0012

±

0.0168

0

0.0009

±

0.0152

0.0005

±

0.0084

0.0011

±

0.0154

0.0024

±

0.0255

0.0120

±

0.0548

0.0034

±

0.0274

0.0052

±

0.0349

0.0204

±

0.0838

0.8317

±

0.2357

0.0059

±

0.0381

0.0017

±

0.0177

0.0124

±

0.0570

0.0006

±

0.0086

0

0.0096

±

0.0496

0.0012

±

0.0196

0.0022

±

0.0265

0.0007

±

0.0101

0.0032

±

0.0306

0.0025

±

0.0254

0.0584

±

0.1232

0.0317

±

0.1051

0.0234

±

0.0844

0.8915

±

0.2095

0.0142

±

0.0561

0.0172

±

0.0657

0.0047

±

0.0301

0

0.0013

±

0.0149

0.0000

±

0.0000

0.0053

±

0.0362

0.0029

±

0.0292

0.0038

±

0.0304

0.0021

±

0.0192

0.0073

±

0.0379

0.0203

±

0.0871

0.0054

±

0.0422

0.0270

±

0.1023

0.8915

±

0.1929

0.0245

±

0.0813

0.0006

±

0.0111

0

0.0009

±

0.0152

0.0009

±

0.0147

0.0002

±

0.0045

0.0000

±

0.0000

0.0075

±

0.0468

0.0001

±

0.0022

0.0015

±

0.0194

0.0417

±

0.1174

0.0404

±

0.1140

0.0374

±

0.1154

0.0707

±

0.1430

0.8913

±

0.1963

0.0028

±

0.0234

0

0.0000

±

0.0000

0.0009

±

0.0147

0.0004

±

0.0091

0.0000

±

0.0000

0.0085

±

0.0490

0.0017

±

0.0163

0.0196

±

0.0682

0.0029

±

0.0318

0.0010

±

0.0124

0.0022

±

0.0261

0.0008

±

0.0199

0.0014

±

0.0240

0.8567

±

0.2211

0

0.0000

±

0.0000

0.0132

±

0.0553

0.0021

±

0.0190

0.0000

±

0.0000

0.0036

±

0.0347

0.0362

±

0.0956

0.0015

±

0.0221

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.0022

±

0.0318

0.0039

±

0.0397

0.0039

±

0.0384

0.0000

±

0.0000

0.0000

±

0.0000

0.0002

±

0.0033

0

0.9417

±

0.1556

0.0011

±

0.0139

0.0080

±

0.0449

0.0029

±

0.0292

0.0000

±

0.0000

0.0013

±

0.0142

0.0000

±

0.0000

0.0022

±

0.0303

0.0051

±

0.0408

0.0028

±

0.0322

0.0021

±

0.0241

0.0020

±

0.0286

0.0326

±

0.1039

0

0.0018

±

0.0304

0.9398

±

0.1521

0.0029

±

0.0273

0.0012

±

0.0176

0.0000

±

0.0000

0.0086

±

0.0436

0.0007

±

0.0111

0.0023

±

0.0257

0.0050

±

0.0450

0.0035

±

0.0350

0.0034

±

0.0377

0.0028

±

0.0364

0.0052

±

0.0451

0

0.0288

±

0.1063

0.0035

±

0.0344

0.9483

±

0.1359

0.0230

±

0.0870

0.0000

±

0.0000

0.0013

±

0.0148

0.0015

±

0.0221

0.0023

±

0.0291

0.0011

±

0.0168

0.0044

±

0.0380

0.0015

±

0.0241

0.0026

±

0.0361

0.0016

±

0.0176

0

0.0077

±

0.0550

0.0034

±

0.0343

0.0221

±

0.0841

0.9657

±

0.1108

0.0000

±

0.0000

0.0011

±

0.0151

0.0004

±

0.0060

0.0186

±

0.0819

0.0012

±

0.0170

0.0028

±

0.0324

0.0042

±

0.0357

0.0047

±

0.0430

0.0049

±

0.0455

0

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.0000

±

0.0000

0.9515

±

0.1258

0.0041

±

0.0339

0.0000

±

0.0000

0.0056

±

0.0412

0.0017

±

0.0180

0.0040

±

0.0360

0.0013

±

0.0193

0.0048

±

0.0455

0.0879

±

0.1680

0

0.0031

±

0.0364

0.0338

±

0.1139

0.0055

±

0.0451

0.0000

±

0.0000

0.0081

±

0.0502

0.9355

±

0.1413

0.0015

±

0.0221

0.0135

±

0.0673

0.0754

±

0.1491

0.0101

±

0.0622

0.0019

±

0.0225

0.0249

±

0.0998

0.0008

±

0.0105

0

0.0043

±

0.0354

0.0017

±

0.0295

0.0018

±

0.0205

0.0012

±

0.0176

0.0018

±

0.0246

0.0022

±

0.0265

0.9024

±

0.1813

Class 1

Class 2

Class 3

Class 4

Class 5

Class 6

Class 7

Class 8

Class 9

Class 10

Class 11

Class 12

Class 13

Class 14

0.4544

±

0.3787

0.0252

±

0.0056

0.0262

±

0.0106

0.0169

±

0.0128

0.0193

±

0.0111

0.0188

±

0.0206

0.0253

±

0.0062

0.0155

±

0.0128

0.0299

±

0.0031

0.0164

±

0.0130

0.0483

±

0.0373

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

0.0191

±

0.0140

0.2676

±

0.1421

0.0252

±

0.0110

0.0175

±

0.0141

0.0193

±

0.0111

0.0155

±

0.0125

0.0253

±

0.0062

0.0144

±

0.0103

0.0299

±

0.0031

0.0164

±

0.0130

0.0300

±

0.0064

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

0.0238

±

0.0304

0.0252

±

0.0056

0.2823

±

0.2983

0.0169

±

0.0128

0.0193

±

0.0111

0.0155

±

0.0125

0.0253

±

0.0062

0.0144

±

0.0103

0.0299

±

0.0031

0.0164

±

0.0130

0.0300

±

0.0064

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

0.0191

±

0.0140

0.0272

±

0.0096

0.0252

±

0.0110

0.3947

±

0.3474

0.0242

±

0.0219

0.0305

±

0.0459

0.0325

±

0.0149

0.0159

±

0.0149

0.0299

±

0.0031

0.0188

±

0.0173

0.0300

±

0.0064

0.0237

±

0.0161

0.0888

±

0.0771

0.0238

±

0.0119

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0185

±

0.0160

0.3955

±

0.3112

0.0214

±

0.0260

0.0253

±

0.0062

0.0144

±

0.0103

0.0299

±

0.0031

0.0182

±

0.0175

0.0300

±

0.0064

0.0207

±

0.0082

0.0252

±

0.0079

0.0238

±

0.0119

0.0255

±

0.0244

0.0413

±

0.0238

0.0292

±

0.0141

0.0222

±

0.0204

0.0194

±

0.0109

0.5248

±

0.3333

0.0344

±

0.0219

0.0144

±

0.0102

0.0299

±

0.0031

0.0165

±

0.0129

0.0322

±

0.0056

0.0204

±

0.0069

0.0275

±

0.0201

0.0255

±

0.0140

0.0204

±

0.0146

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0207

±

0.0137

0.0155

±

0.0125

0.1999

±

0.0965

0.0144

±

0.0103

0.0299

±

0.0031

0.0164

±

0.0130

0.0300

±

0.0064

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

0.0199

±

0.0146

0.0302

±

0.0188

0.0252

±

0.0110

0.0169

±

0.0128

0.0209

±

0.0148

0.0274

±

0.0557

0.0253

±

0.0062

0.4674

±

0.2631

0.0429

±

0.0199

0.0166

±

0.0130

0.0322

±

0.0056

0.0208

±

0.0081

0.0215

±

0.0109

0.0238

±

0.0119

0.0191

±

0.0139

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0193

±

0.0111

0.0186

±

0.0201

0.0253

±

0.0062

0.0229

±

0.0144

0.1166

±

0.0541

0.0165

±

0.0129

0.0300

±

0.0064

0.0238

±

0.0093

0.0215

±

0.0109

0.0239

±

0.0119

0.0191

±

0.0140

0.0274

±

0.0113

0.0252

±

0.0110

0.0376

±

0.0560

0.0319

±

0.0258

0.0175

±

0.0155

0.0477

±

0.0461

0.0144

±

0.0102

0.0369

±

0.0183

0.4892

±

0.3606

0.0300

±

0.0064

0.0242

±

0.0095

0.0215

±

0.0109

0.0299

±

0.0284

0.0255

±

0.0346

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0214

±

0.0141

0.0164

±

0.0145

0.0253

±

0.0062

0.0145

±

0.0102

0.0299

±

0.0031

0.0176

±

0.0125

0.1314

±

0.1014

0.0248

±

0.0143

0.0215

±

0.0109

0.0238

±

0.0119

0.0200

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0273

±

0.0348

0.0231

±

0.0297

0.0163

±

0.0125

0.0253

±

0.0062

0.0172

±

0.0220

0.0299

±

0.0031

0.0175

±

0.0155

0.0339

±

0.0142

0.1887

±

0.0792

0.0391

±

0.0179

0.0337

±

0.0251

0.0202

±

0.0139

0.0272

±

0.0096

0.0262

±

0.0106

0.0283

±

0.0355

0.0196

±

0.0111

0.0164

±

0.0129

0.0325

±

0.0149

0.0172

±

0.0220

0.0299

±

0.0031

0.0175

±

0.0155

0.0339

±

0.0142

0.1654

±

0.0701

0.2576

±

0.1979

0.0337

±

0.0251

0.0219

±

0.0217

0.0252

±

0.0056

0.0252

±

0.0110

0.0182

±

0.0161

0.0204

±

0.0119

0.0155

±

0.0125

0.0253

±

0.0062

0.0146

±

0.0104

0.0440

±

0.0234

0.0250

±

0.0473

0.0341

±

0.0152

0.0446

±

0.0225

0.0282

±

0.0229

0.2792

±

0.2911

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0600

±

0.0933

0.0206

±

0.0118

0.0158

±

0.0124

0.0253

±

0.0062

0.0144

±

0.0103

0.0299

±

0.0031

0.0171

±

0.0142

0.0300

±

0.0064

0.0198

±

0.0051

0.0215

±

0.0109

0.0239

±

0.0119

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0216

±

0.0299

0.0405

±

0.0417

0.0156

±

0.0124

0.0571

±

0.0172

0.0212

±

0.0122

0.0299

±

0.0031

0.0240

±

0.0327

0.0423

±

0.0268

0.0557

±

0.0272

0.0378

±

0.0235

0.0330

±

0.0209

0.0254

±

0.0205

0.0354

±

0.0170

0.0501

±

0.0335

0.0282

±

0.0305

0.0267

±

0.0238

0.0158

±

0.0123

0.0253

±

0.0062

0.0813

±

0.0913

0.0299

±

0.0031

0.0168

±

0.0131

0.0300

±

0.0064

0.0548

±

0.0337

0.0314

±

0.0201

0.0303

±

0.0186

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0292

±

0.0259

0.0199

±

0.0217

0.0325

±

0.0149

0.0144

±

0.0103

0.0369

±

0.0183

0.0233

±

0.0394

0.0339

±

0.0142

0.0256

±

0.0206

0.0275

±

0.0201

0.0243

±

0.0113

0.0191

±

0.0140

0.0252

±

0.0056

0.0305

±

0.0215

0.0264

±

0.0418

0.0206

±

0.0139

0.0179

±

0.0174

0.0521

±

0.0579

0.0144

±

0.0103

0.0299

±

0.0031

0.0181

±

0.0165

0.0341

±

0.0152

0.0224

±

0.0119

0.0424

±

0.0268

0.0238

±

0.0119

0.0191

±

0.0140

0.0388

±

0.0232

0.0252

±

0.0110

0.0463

±

0.0652

0.0241

±

0.0156

0.0186

±

0.0170

0.0313

±

0.0102

0.0579

±

0.0319

0.0299

±

0.0031

0.0189

±

0.0176

0.0300

±

0.0064

0.0205

±

0.0084

0.0282

±

0.0229

0.0297

±

0.0320

0.0191

±

0.0140

0.0252

±

0.0056

0.0254

±

0.0106

0.0169

±

0.0128

0.0193

±

0.0111

0.0155

±

0.0125

0.0253

±

0.0062

0.0144

±

0.0103

0.0299

±

0.0031

0.0164

±

0.0130

0.0300

±

0.0064

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0173

±

0.0129

0.0294

±

0.0225

0.0164

±

0.0129

0.0253

±

0.0062

0.0243

±

0.0230

0.0499

±

0.0226

0.0198

±

0.0165

0.0300

±

0.0064

0.0246

±

0.0100

0.0215

±

0.0109

0.0517

±

0.0447

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0193

±

0.0111

0.0155

±

0.0125

0.0253

±

0.0062

0.0144

±

0.0103

0.0299

±

0.0031

0.0177

±

0.0184

0.0300

±

0.0064

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0193

±

0.0111

0.0155

±

0.0125

0.0253

±

0.0062

0.0144

±

0.0103

0.0369

±

0.0183

0.0164

±

0.0130

0.0341

±

0.0152

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0193

±

0.0111

0.0155

±

0.0125

0.0253

±

0.0062

0.0144

±

0.0103

0.0299

±

0.0031

0.0164

±

0.0130

0.0300

±

0.0064

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0193

±

0.0111

0.0162

±

0.0136

0.0253

±

0.0062

0.0144

±

0.0103

0.0299

±

0.0031

0.0177

±

0.0157

0.0300

±

0.0064

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0193

±

0.0111

0.0162

±

0.0136

0.0253

±

0.0062

0.0144

±

0.0102

0.0377

±

0.0217

0.0421

±

0.0480

0.0300

±

0.0064

0.0208

±

0.0079

0.0215

±

0.0109

0.0238

±

0.0119

0.0191

±

0.0140

0.0252

±

0.0056

0.0252

±

0.0110

0.0169

±

0.0128

0.0193

±

0.0111

0.0155

±

0.0125

0.0253

±

0.0062

0.0144

±

0.0103

0.0299

±

0.0031

0.0164

±

0.0130

0.0300

±

0.0064

0.0198

±

0.0051

0.0215

±

0.0109

0.0238

±

0.0119

Class 15

Class 16

Class 17

Class 18

Class 19

Class 20

Class 21

Class 22

Class 23

Class 24

Class 25

Class 26

Class 27

Class 28

0.0191

±

0.0123

0.0251

±

0.0084

0.0213

±

0.0378

0.0231

±

0.0121

0.0251

±

0.0076

0.0220

±

0.0145

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0191

±

0.0123

0.0251

±

0.0084

0.0127

±

0.0109

0.0231

±

0.0121

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0191

±

0.0123

0.0251

±

0.0084

0.0125

±

0.0110

0.0231

±

0.0121

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0375

±

0.0328

0.0276

±

0.0102

0.0152

±

0.0146

0.0231

±

0.0121

0.0368

±

0.0325

0.0353

±

0.0384

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0191

±

0.0123

0.0362

±

0.0226

0.0159

±

0.0184

0.0270

±

0.0197

0.0251

±

0.0076

0.0244

±

0.0119

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0318

±

0.0107

0

0.0191

±

0.0123

0.0327

±

0.0201

0.0125

±

0.0110

0.0243

±

0.0125

0.0251

±

0.0076

0.0298

±

0.0238

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0191

±

0.0123

0.0251

±

0.0084

0.0130

±

0.0117

0.0231

±

0.0121

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0223

±

0.0188

0.0327

±

0.0201

0.0207

±

0.0311

0.0231

±

0.0121

0.0251

±

0.0076

0.0237

±

0.0132

0

0.0444

±

0.0186

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0196

±

0.0117

0.0251

±

0.0084

0.0144

±

0.0184

0.0233

±

0.0118

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0197

±

0.0115

0.0486

±

0.0413

0.0125

±

0.0109

0.0243

±

0.0117

0.0251

±

0.0076

0.0257

±

0.0113

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0314

±

0.0116

0

0.0193

±

0.0121

0.0251

±

0.0084

0.0125

±

0.0109

0.0231

±

0.0121

0.0364

±

0.0144

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0241

±

0.0173

0.0284

±

0.0133

0.0167

±

0.0253

0.0264

±

0.0210

0.0251

±

0.0076

0.0205

±

0.0149

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0301

±

0.0079

0

0.0241

±

0.0173

0.0284

±

0.0133

0.0216

±

0.0326

0.0264

±

0.0210

0.0291

±

0.0164

0.0205

±

0.0149

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0301

±

0.0079

0

0.0302

±

0.0192

0.0251

±

0.0084

0.0128

±

0.0110

0.0257

±

0.0177

0.0251

±

0.0076

0.0220

±

0.0145

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.3424

±

0.3057

0.0302

±

0.0115

0.0129

±

0.0111

0.0231

±

0.0121

0.0251

±

0.0076

0.0538

±

0.0608

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0336

±

0.0305

0.2156

±

0.1776

0.0227

±

0.0284

0.0380

±

0.0345

0.0251

±

0.0076

0.0226

±

0.0135

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0328

±

0.0121

0

0.0227

±

0.0141

0.0251

±

0.0084

0.5931

±

0.3144

0.0237

±

0.0124

0.0251

±

0.0076

0.0557

±

0.0654

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0262

±

0.0293

0.0579

±

0.0455

0.0130

±

0.0119

0.2965

±

0.3181

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0444

±

0.0186

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0193

±

0.0121

0.0251

±

0.0084

0.0171

±

0.0236

0.0510

±

0.0434

0.2839

±

0.1802

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0830

±

0.0987

0.0251

±

0.0084

0.0236

±

0.0220

0.0266

±

0.0179

0.0364

±

0.0144

0.3397

±

0.3202

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0191

±

0.0123

0.0251

±

0.0084

0.0125

±

0.0110

0.0231

±

0.0121

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0272

±

0.0275

0.0294

±

0.0185

0.0149

±

0.0176

0.0287

±

0.0207

0.0251

±

0.0076

0.0202

±

0.0153

0

0.1207

±

0.0506

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0191

±

0.0123

0.0251

±

0.0084

0.0132

±

0.0131

0.0231

±

0.0121

0.0251

±

0.0076

0.0220

±

0.0145

0

0.0305

±

0.0054

0.0865

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0313

±

0.0115

0

0.0191

±

0.0123

0.0251

±

0.0084

0.0125

±

0.0110

0.0231

±

0.0121

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0915

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0191

±

0.0123

0.0251

±

0.0084

0.0125

±

0.0110

0.0284

±

0.0209

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0444

±

0.0186

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

0.0193

±

0.0121

0.0251

±

0.0084

0.0125

±

0.0110

0.0231

±

0.0121

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0318

±

0.0000

0.0336

±

0

0

0.0898

±

0.0026

0.0336

±

0.0121

0

0.0193

±

0.0121

0.0303

±

0.0188

0.0125

±

0.0110

0.0239

±

0.0137

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0305

±

0.0054

0.0865

±

0.0000

0.0336

±

0

0

0.0513

±

0.0273

0.2045

±

0.1652

0

0.0191

±

0.0123

0.0251

±

0.0084

0.0125

±

0.0110

0.0284

±

0.0209

0.0251

±

0.0076

0.0202

±

0.0153

0

0.0444

±

0.0186

0.0318

±

0.0000

0.0336

±

0

0

0.0330

±

0.0009

0.0287

±

0.0067

0

created from chemical compound of each drug using their SMILES notations. SMILES (Simplified Molecular Input Line Entry System) is a chemical notation that represents a chemical structure as a string so it can be used by the computer. Table 5 lists examples

Class 1

Class 2

Class 3

Class 4

Class 5

Class 6

Class 7

Class 8

Class 9

0.6166

±

0.2722

0.0528

±

0.0882

0.0097

±

0.0450

0.1056

±

0.1204

0.0255

±

0.0774

0.0109

±

0.0521

0.0000

±

0.0000

0.0324

±

0.0685

0.0241

±

0.0701

0.0938

±

0.1400

0.6153

±

0.2831

0.0196

±

0.0634

0.1046

±

0.1170

0.0117

±

0.0485

0.0000

±

0.0000

0.0000

±

0.0000

0.0585

±

0.0976

0.0144

±

0.0515

0.0290

±

0.0848

0.0409

±

0.0934

0.7113

±

0.2697

0.0235

±

0.0497

0.0021

±

0.0189

0.0109

±

0.0521

0.0000

±

0.0000

0.0368

±

0.0771

0.0074

±

0.0320

0.1675

±

0.1790

0.1583

±

0.1725

0.1130

±

0.1811

0.6573

±

0.2638

0.0212

±

0.0616

0.0217

±

0.0720

0.0000

±

0.0000

0.0695

±

0.1109

0.0144

±

0.0515

0.0267

±

0.0816

0.0198

±

0.0755

0.0117

±

0.0629

0.0196

±

0.0633

0.8670

±

0.2324

0.0000

±

0.0000

0.0000

±

0.0000

0.0034

±

0.0205

0.0028

±

0.0167

0.0053

±

0.0338

0.0095

±

0.0469

0.0172

±

0.0755

0.0071

±

0.0458

0.0076

±

0.0574

0.8261

±

0.2435

0.0000

±

0.0000

0.0059

±

0.0302

0.0028

±

0.0167

0.0011

±

0.0199

0.0017

±

0.0208

0.0014

±

0.0168

0.0014

±

0.0221

0.0000

±

0.0000

0.0217

±

0.1043

0.8750

±

0.2315

0.0025

±

0.0169

0.0000

±

0.0000

0.0481

±

0.1050

0.0830

±

0.1300

0.0686

±

0.1510

0.0671

±

0.1061

0.0446

±

0.1255

0.0652

±

0.1722

0.0625

±

0.1768

0.7568

±

0.2872

0.0069

±

0.0417

0.0118

±

0.0523

0.0186

±

0.0631

0.0475

±

0.1352

0.0139

±

0.0475

0.0203

±

0.0970

0.0435

±

0.1441

0.0625

±

0.1768

0.0343

±

0.1022

0.9273

±

0.1932

Class 1

Class 2

Class 3

Class 4

Class 5

Class 6

Class 7

Class 8

Class 9

Class 10

0

.

6855

±

0

.

2143

0

.

0069

±

0

.

0475

0

.

0273

±

0

.

0792

0

.

0759

±

0

.

1119

0

.

0427

±

0

.

0949

0

.

0305

±

0

.

0699

0

.

0323

±

0

.

0884

0

.

0063

±

0

.

0449

0

.

0718

±

0

.

1116

0

.

0209

±

0

.

0564

0

.

0064

±

0

.

0137

0

.

6357

±

0

.

2011

0

.

1042

±

0

.

1345

0

.

0234

±

0

.

0492

0

.

0522

±

0

.

1020

0

.

0321

±

0

.

1098

0

.

0388

±

0

.

0742

0

.

0358

±

0

.

0803

0

.

0471

±

0

.

0975

0

.

0244

±

0

.

0843

0

.

0355

±

0

.

0686

0

.

0039

±

0

.

0320

0

.

6177

±

0

.

1745

0

.

0645

±

0

.

1026

0

.

0912

±

0

.

1208

0

.

0065

±

0

.

0253

0

.

0355

±

0

.

0873

0

.

0109

±

0

.

0489

0

.

1106

±

0

.

1440

0

.

0236

±

0

.

0643

0

.

0715

±

0

.

1133

0

.

0006

±

0

.

0068

0

.

0319

±

0

.

0893

0

.

6724

±

0

.

2079

0

.

0339

±

0

.

0869

0

.

0226

±

0

.

0765

0

.

0485

±

0

.

1025

0

.

0037

±

0

.

0285

0

.

0968

±

0

.

1271

0

.

0181

±

0

.

0581

0

.

0655

±

0

.

0964

0

.

0249

±

0

.

0950

0

.

0470

±

0

.

1001

0

.

0588

±

0

.

1016

0

.

5907

±

0

.

1826

0

.

0200

±

0

.

0496

0

.

0581

±

0

.

1077

0

.

0296

±

0

.

0853

0

.

0837

±

0

.

1143

0

.

0216

±

0

.

0638

0

.

0807

±

0

.

1251

0

.

0151

±

0

.

0787

0

.

0123

±

0

.

0571

0

.

0039

±

0

.

0107

0

.

0805

±

0

.

1292

0

.

6902

±

0

.

2192

0

.

0186

±

0

.

0581

0

.

0005

±

0

.

0028

0

.

0981

±

0

.

1394

0

.

0000

±

0

.

0000

0

.

0651

±

0

.

0921

0

.

0120

±

0

.

0552

0

.

0448

±

0

.

0999

0

.

0553

±

0

.

1193

0

.

0775

±

0

.

1190

0

.

0187

±

0

.

0587

0

.

6171

±

0

.

2053

0

.

0084

±

0

.

0363

0

.

0528

±

0

.

0863

0

.

0483

±

0

.

0986

0

.

0198

±

0

.

0375

0

.

0092

±

0

.

0419

0

.

0227

±

0

.

0631

0

.

0445

±

0

.

0812

0

.

1065

±

0

.

1305

0

.

0034

±

0

.

0095

0

.

0344

±

0

.

0755

0

.

6798

±

0

.

2135

0

.

0597

±

0

.

0909

0

.

0201

±

0

.

0542

0

.

0683

±

0

.

1106

0

.

0044

±

0

.

0361

0

.

0119

±

0

.

0543

0

.

1223

±

0

.

1512

0

.

0584

±

0

.

1106

0

.

0282

±

0

.

0669

0

.

0183

±

0

.

0592

0

.

0058

±

0

.

0321

0

.

6759

±

0

.

2015

0

.

0066

±

0

.

0316

0

.

0478

±

0

.

0898

0

.

0106

±

0

.

0573

0

.

0225

±

0

.

0564

0

.

1027

±

0

.

1356

0

.

0618

±

0

.

1078

0

.

0305

±

0

.

0705

0

.

0909

±

0

.

1285

0

.

0019

±

0

.

0103

0

.

0571

±

0

.

0908

0

.

5740

±

0

.

1959

of three drugs' names, IDs, and their SMILES strings. Because SMILES describes drug's chemical compound, We use RDKit library [36] to convert the chemical compound to 191 measures(scalars) of that chemical compound and use it as drug node (target node) features. To obtain the label, we use the MeSH-tree parsing tool [37] to convert the disease ID to corresponding MeSH prefix code which corresponds to high level disease and obtain in total 28 high-level diseases that drugs could potentially target on. Table 6 lists the 28 labels used in DRUG dataset (including MeSH prefix codes and names).

URBAN is a functional urban area network constructed based on the city of Porto, Portugal, uniformly divided into a grid of 30 by 80 blocks. A dataset comprising 22,208 points of interest (POI) is sourced from the OpenStreetMap platform 2 to generate the label distribution. This dataset includes a large amount of fundamental categories like hotels, banks and churches, which have been further categorized into 10 more general groups, as detailed in Table 4. We count the types and corresponding numbers of POIs to generate the label for each block. After excluding blocks devoid of any POIs, the remaining blocks were primarily divided into four different types of nodes using the Louvain algorithm [38]. These node types are named as Nature Residence, Comprehensive Service, Green Leisure and Transit Junction based on their main functions and nodes from Nature Residence are our target nodes. The edges are constructed by Taxi Service Trajectory 3 , which documents the taxi routines (from origins to destinations) in Porto. In our resulting heterogeneous graph, two target nodes are considered neighbors if they have the same recorded destination.

node (e.g. Class 7 of the ACM dataset in Table 8, or Classes 21, 25, and 28 in the DRUG dataset in Table 10). In this case, the average label distributions of the class are reported as 0 in the tables.

D.2 Baselines

We extend existing heterogeneous graph learning methods to integrate KL-divergence loss, and implement three baselines for comparisons. In addition, three variants of the proposed HGDL method are compared for ablation study purposes:

GCNKL: A baseline uses graph constructed from each meta-path to train a vanilla GCN, using KL-divergence as loss function, and reports the best meta-path result. This baseline is used to demonstrate whether the proposed HGDL can outperform the best single meta-path result.

HANKL: This baseline is to compare HGDL 's semantic fusion component with HAN [25], a classical heterogeneous network learning semantic fusion method. HANKL uses HAN to integrate embedding from meta-path graphs created from different meta-paths, and uses KL-divergence as the loss function for training.

SeHGNNKL This baseline is to compare HGDL 's semantic fusion with SeHGNN [27], a state of the art transformer based heterogeneous semantic fusion learning method. SeHGNNKL uses neighbor aggregation proposed by SeHGNN to aggregate meta-paths embedding, with KL-divergence being used as the loss function for training.

HGDL: The proposed HGDL method.

HGDL ¬ TH: This is a variant of the HGDL for ablation study. HGDL ¬ TH removes the graph topology homogenization module (Sec 4.1). The embedding is trained from each meta-path with topology and feature consistence-aware graph transformer (Sec 4.2) being applied to each meta-path. The result of the best meta-path is reported, and the purpose is to demonstrate HGDL's topology homogenization effectiveness.

HGDL ¬ transformer : This is a variant of the HGDL for ablation study. HGDL ¬ transformer uses GCN, instead of the transformer (Sec 4.2), to learn embeddings. This serves as an ablation study to show the advantage of HGDL's transformer component for embedding learning.

HGDLED This is a variant of the HGDL for ablation study. HGDLED replaces HGDL's topology and feature consistence-aware graph transformer (Sec. 4.2) using random edge dropout method [31]. The purposes to demonstrate the advantage of consistent-aware graph transfer, which resembles to learnable edge dropping, compared to random edge dropout.

D.3 Experiment Setup

All results reported in the paper are based on 5 times average for each method. Hyperparameters of all the baselines include the hidden dimension for the embedding learning and the hidden dimension for the semantic fusion learning (if the semantic fusion part exists), and the learning rate. In practice, a 0.005 learning rate is fixed for all the baselines unless it is observed that specific method cannot converge under such learning rate or can converge much faster with a large learning rate. An early stop patience signal is set and a maximum number of epochs is fixed. In practice, we observed convergence before reaching maximum number of epochs in general. For the methods that cannot learn well with all the hyperparameter search, we report the best results we got. For the HDGLED method, the originally learned feature topology for HGDL is replaced by a 0-1 matrix generated by a Bernoulli distribution with a drop rate provided to convert to a 0-1 matrix. The drop rate for the HDGLED baseline in Table 2 in the main manuscript is 0.1. Fig 5 provides the results of different drop rate cases. For HGDL we have additional hyperparameters negative slope of Leaky Relu activation and γ for regularization loss.

The train:validation:test splits for DRUG, ACM, DBLP, YELP, and URBAN dataset are 8:1:1, 7:1:2, 4:1:5, 8:1:1, 5:3:2, respectively. We used a smaller training set (40%) for DBLP because because this dataset has comprehensive nodal features (8,920 features) which are the-bag-of-words of papers published by authors. DBLP also has much less number of labels (4 classes), compared to other datasets. The rich nodal features, combined with less number of classes, provide rather rich information to derive research fields. So we reduce the training set size, compared to the test set, to provide less label information for learning.

D.4 Evaluation metric

Given one predicted label distribution ˆ y and its corresponding ground truth distribution y , the six metrics used in the Table 2 can be defined as the following:

Cosine Distance(COD):

Canberra Distance(CAD):

Chebyshev Distance(CHD):

Clark Distance (CLD):

Intersection Score(IND):

Kullback-Leibler Divergence(KL):

E Supplement C: Additional Results and Analysis

E.1 Edge Dropping Comparisons

HGDL treats the learnt feature attention as another adjacency matrix, and incorporates it into the graph adjacency matrix by a ⊙ operator, which provides an intersection of neighbor sets for a node, i.e. , the resulting convolution layer is propagated to a neighbor set that both feature and graph topology agrees on. This operation is similar to the recently proposed drop message idea that randomly masks edges for the nodes. Instead of random masking, HGDL selects the neighbors that both feature and graph topology agree on. In other words, Eq. (2) in the main manuscript functions similarly to the edge dropout [31]; in lieu of randomly removing edges, we enforce a neighbor-set intersection, where the information is only propagated from the neighbors on which the feature space and meta-path topology both agree.

To compare the performance between our approach vs. the edge dropout [31], the variant HGDLED is used to replace our transformer based approach by using random edge dropout. In this case, for HDGLED, the originally learned feature topology for HGDL is replaced by a 0-1 matrix generated by a Bernoulli distribution with a drop rate provided to convert to a 0-1 matrix. For comparisions, we vary the edge drop rates and report HGDLED's performance in Figure 5. The results show that that HGDLED with random edge dropping doesn't necessarily help improve the distribution learning, implying that our method works not just by relying on the decreasing of the edge number, but also relying on choosing the proper edges to drop.

E.2 Validation Loss Comparisons

Figures 6, 7, and 8 report the training process validation loss ( w.r.t. KL-divergence loss) between HANKL, SeHGNNKL, and HGDL . Because early stop is applied to all methods, they terminate at respective number of epochs. In our experiments, a patience score is used to determine the minimum loss. A method will continue if a smaller validation loss is found within patience number of epochs after the current smallest loss, and terminates otherwise.

ACM

DRUG

4

loss

2

0.0

0.2

0.4

0.6

0.8

edgedroprate

1.4

HANKL

HGDL

1.2

SeHGNNKL

Validation Loss

1.0

0.8

0.6

0

100

200

300

400

500

600

700

800

Epochs

1.4

HANKL

1.2

HGDL

SeHGNNkL

Validation Loss

1.0

0.8

0.6

0.4

0.2

0.0

0

500

1000

1500

2000

2500

3000

Epochs

HANKL

HGDL

6

SeHGNNkL

ValidationLoss

5

4

3

2

0

50

100

150

200

250

300

350

Epochs

bnp

2.0

urban

yelp

1.5

diqp

KL Divergence

1.0

0.5

0.0

-4

-3

The comparisons show that HGDL achieves the smallest KL-divergence loss across all datasets. For the DRUG dataset (Figure 6), HGDL converges at much better optimal point than the other two baselines. This is also evident by the results shown in the Table 2 of the main manuscript (which are validated on the test set nodes).

E.3 Sensitive Analysis of hyperparameter

To study the effect of hyperparameter γ on the model performance, we explore the γ value ranging from [0,1e-5,1e-4,1e-3,1e-2,0.1] and see how the KL divergence metric is affected by different γ values for our five datasets shown in Figure 9. We can observe that optimal γ values vary for different datasets and therefore it is important to search for a proper γ value for each datasets.

E.4 Comparison against HINormer and GLDL

HINormer [39] is a recent heterogenous graph learning baseline and GLDL method is a recent LDL learning method in homogeneous graphs. To make a more comprehensive comparisons, Table 13 provides the results applied to DBLP and Drug dataset with these two additional baselines. We observe that our method is still competitive in most metrics in terms of GLDL and is more superior to HINormer method in general. This further validates the superiority of our design on heterogenous settings.

F Supplement D: Proof of Theorem 1

Theorem 3. Let E [ L ( X,A i ) ( ℏ )] be the empirical risk of using the i -th meta-path graph A i ∈ A for label distribution prediction. With the SGD step-size η , we have

Proof. We define the cumulative risk L ( X,A i ) ( ℏ ) = ∑ n i =1 [ ℓ ( ℏ ( X,A i )[ i ] , y i )] over n training iterations, and a quantitative variable Q n as follows:

where it is trivial to derive Q 1 = k , as no risk will be suffered if no target node attends training. By performing stochastic gradient descent (SGD) over n iterations, we have

and

where ℓ i (ˆ y n , y n ) = ℓ ( ℏ ( X,A i )[ n ] , y n ) denotes the immediate loss evaluated at the n -th training iteration based on the i -th meta-path graph topology. The attention matrix Θ are with all entries non-negative by its definition. To proceed, we introduce the lemma by [40].

Lemma 4 (Hoeffding Inequality) . Let X be a random variable with a ≤ X ≤ b . Then for any s ∈ R ,

With normalized immediate loss ℓ i (ˆ y n , y n ) ∈ [0 , 1] , ∀ i , we can relax Eq. (12) based on the convexity of ℓ and ℏ :

We can observe that over n training iterations:

Chaining Eq. (11) and Eq. (14) we arrive at:

and dividing the both sides by n , we have

Note that E [ L ( X,A i ) ( ℏ )] = 1 n L ( X,A i ) ( ℏ ) concludes the proof.

G Supplement E: Proof of Theorem 2

Theorem 5. Let ℏ ∈ H : X ×G → R q be an l -layer message-passing neural network with maximum hidden dimension k , of which the i -th layer is parameterized by W i . Then for any δ, γ, B > 0 and

l > 1 , with probability at least 1 -δ we have

where D ( W i ) = ∏ l i =1 ∥ W i ∥ 2 2 · ∑ l i =1 ( ∥ W i ∥ 2 F / ∥ W i ∥ 2 2 ) bounds the hypothesis space and b is a constant.

Proof We first try to explore the boundary of a GCN without label distribution, and use the multiclass γ -margin loss following [41, 42]. The generalization error and the empirical error are defined as,

̸

̸

where γ > 0 and f w ( X,A ) is the l -th layer representations, i.e., H l = f w ( X,A ) . 1 is a all-one vector. Two necessary lemmas are derived from this. First, a margin-based generalization bound is given to guarantee that, as long as the change of the output brought by the perturbations is small with a large probability, the corresponding generalization bound [42] is defined as:

Lemma 6. Let f w ( x ) : X → R K be any model with parameters w , and let P be any distribution on the parameters that is independent of the training data. For any w , we construct a posterior Q ( w + u ) by adding any random perturbation u to w , s.t., P ( max x ∈X | f w + u ( x ) -f w ( x ) | ∞ < γ 4 ) > 1 2 . Then, for any γ, δ > 0 , with probability at least 1 -δ we have:

where n is the number of instances of training set. Hence, in order to apply Lemma 6 , we must ensure that the change of the output brought by the weight perturbations is small with a large probability. In the following lemma, we bound this change using the product of the spectral norms of learned weights at each layer and a term depending on some statistics of the graph [43].

Lemma 7. For any B > 0 , l > 1 , let f w ∈ H : X × G → R K be a l-layer GCN. Then for any w , and x ∈ X B,h 0 , and any perturbation u = vec ( { U i } l i =1 ) such that ∀ i ∈ N + l , ∥ U i ∥ 2 ≤ 1 l ∥ W i ∥ 2 , the change in the output of GCN is bounded as,

Then, we can deduce the bound for a GCN without label distribution. First, let β = ( ∏ l i =1 ∥ W i ∥ 2 ) 1 /t . Weights are normalized as ˜ W i = β ∥ W i ∥ 2 W i . Due to the homogeneity of ReLU, i.e., aϕ ( x ) = ϕ ( ax ) , ∀ a ≥ 0 . ∏ l i =1 ∥ W i ∥ 2 = ∏ l i =1 ∥ ∥ ∥ ˜ W i ∥ ∥ ∥ 2 and ∥ W i ∥ F / ∥ W i ∥ 2 =

∥ ∥ ∥ ˜ W i ∥ ∥ ∥ F / ∥ ∥ ∥ ˜ W i ∥ ∥ ∥ 2 , i.e., the terms present in the bound remain unchanged after normalization. Therefore, w.l.o.g., we conduct the assumption that the norm is equal across all layers, i.e., ∀ i, ∥ W i ∥ 2 = β . Consider the prior P = N ( 0 , σ 2 I ) and the random perturbation u ∼ N ( 0 , σ 2 I ) . Note that the σ of the prior and the perturbation keep the same and will be set according to β . Specifically, the value of σ is set based on some approximation ˜ β of β since the prior P can not be directly deduced by any learned weights. We select the approximation ˜ β as the cover set that encompasses the possible significant range of β . For the moment, assuming we have a constant ˜ β , we can examine β that

fulfills the condition | β -˜ β | ≤ 1 l β . Note that this also implies

According to [44], for random perturbations U i ∈ R h × h which is object to the distribution U i ∼ N ( 0 , σ 2 I ) ,

We consider the perturbations of all layers, thus:

Setting 2 lke -t 2 / 2 kσ 2 = 1 2 , we can have t = σ √ 2 k log(4 lk ) .

The likelihood that the spectral norm perturbation of any layer not exceeds σ √ 2 h log(4 lk ) is at least 1 2 . Plugging this bound into Lemma 7, we have with probability at least 1 2 ,

where we can set σ = γ 4 e 2 Bd L -1 2 ˜ β l -1 l √ k log(4 lk ) to get a new inequality. Note that Lemma 7 also requires ∀ i ∈ N + l , ∥ U i ∥ 2 ≤ 1 l ∥ W i ∥ 2 . The requirement is satisfied if σ ≤ β l √ 2 k log(4 lk ) which in turn can be satisfied if

since the chosen value of σ satisfies σ ≤ γ 4 eBd l -1 2 β l -1 l √ 2 k log(4 lk ) .

Note that Eq. (22) can be rewritten as γ 4 eB d 1 -1 2 ≤ β l . KL term in the PAC-Bayes bound in Lemma 6 is deduced as:

From Lemma 6, fixing any ˜ β , with probability 1 -δ and for all w such that | β -˜ β | ≤ 1 l β . Let D ( W i ) = ∏ l i =1 ∥ W i ∥ 2 2 · ∑ l i =1 ( ∥ W i ∥ 2 F / ∥ W i ∥ 2 2 ) we can get,

The choice of ˜ β is important, such that for any β , we can bound the generalization error like Eq. (24). First, the value range of β is decided as the following,

since otherwise the bound holds trivially as L ( X,A ) ( f w ) ≤ 1 by definition. Note that the lower bound in Eq. (25) ensures that Eq. (22) holds which in turn justifies the applicability of Lemma 7. If β < 1 √ d ( γ √ d 2 B ) 1 /l , then for any ( X,A ) and any j ∈ N + K , | f ( X,A )[ j ] | ≤ γ 2 . To prove this, we have,

Therefore, we always have L ( X,A ) ( f w ) = 1 when β < 1 √ d ( γ √ d 2 B ) 1 /l . Alternatively, if β > 1 √ d ( 2 √ nd 2 B ) 1 /l , the term inside the big-O notation in Eq. (24) would be,

where we set k ≥ 2 in practice and l ≥ 2 , while considering the fact ∥ W i ∥ F ≥ ∥ W i ∥ 2 . To account for β within the range defined by Eq. (25), a condition ensuring that | β -˜ β | ≤ 1 l β would be

| β -¯ β | ≤ 1 l √ d ( γ √ d 2 B ) 1 /l . So, if we can identify an overlap of the span in Eq. (25) with a radius of 1 l √ d ( ∂ √ d 2 B ) 1 /l and ascertain that bounds akin to Eq. (24) are met when ˜ β assumes any value within that overlap, then a bound that is valid for every β can be established. Clearly, it is only essential to ponder a coverage C of magnitude | C | = l 2 ( n 1 2 -1 ) . Thus, representing the event of Eq. (24) when ˜ β adopts the i -th value from the coverage as E i , we deduce:

Note ¯ E i denotes the complement of E i . Hence, we now present the PAC-Bayes generalization bound [43] of GCNs as Theorem 8.

Theorem 8. For any B > 0 , l > 1 , let f w ∈ H : X × G → R K be a l layer GCN. Then for any δ, γ > 0 , with probability at least 1 -δ over the choice of an i.i.d. size-n training set S according to D , for any w , we have,

Then we consider a GCN with label distribution learning, and explore the bound in term of Rademacher complexity. Specifically, let η ( · ) be a label distribution function, thus η ( x i ) = { η y 1 x i , . . . , η y p x i } . Given a function class H and a loss function ℓ ( · ) , for function ℏ ∈ H , its corresponding risk and empirical risk are defined as L G ( f w ) = E x ∼D [ ℓ ( ℏ ( x ) , η ( x ))] and L ( X,A ) ( f w ) = 1 n ∑ n i =1 ℓ ( ℏ ( x i ) , η ( x i )) , respectively. Recall the definition of rademacher complexity w.r.t. S and ℓ ,

where ϵ 1 , . . . , ϵ n are n independent rademacher random variables with P ( ϵ i = 1) = P ( ϵ i = -1) = 1 / 2 .

Then we can derive the following necessary lemma [45] [46],

Lemma 9. Let H be a family of functions. For a loss function ℓ bounded by µ , then for any δ > 0 , with probability at least 1 -δ , for all ℏ ∈ H such that

and the theorem of Rademacher complexity of the method for loss function KL-divergence [8]:

Theorem 10. Let H be a family of functions for multi-output linear regression, and H j be a family of functions for the j -th output. Rademacher complexity of ME(Maximum-Entropy) with KL loss satisfies

Note that KL( u , · ) is not ρ -Lipschitz over R m for any ρ ∈ R and u ∈ R m . Define function ϕ ( · , · ) as KL( · , SF( · )) . Next we show that ϕ ( u , · ) satisfy Lipschitzness. For p , q ∈ R m ,

which equals

̸

̸

Observing that ln ( 1 + ∑ j exp v i ) is 1-Lipschitz for v ∈ R m , thus right-hand side of preceding equation is bounded by

namely, ϕ is ( √ p +1) -Lipschitz. According to [47], we have:

Although only implementing KL-divergence does not satisfy Lipschitzness, the combination of KL and Softmax(SF) is ( √ p +1) -Lipschitz. Define class of functions of j -th output with weight constraints as H j = { x → w j · x : ∥ w j ∥ 2 ≤ 1 } . According to [48], rademacher complexity of H j satisfies

Then right-hand side of Eq. (31) is bounded as

According to Lemma 9, while we have the rademacher complexity of LDL with KL-divergence loss defined in Eq. (32), the bound can be defined as:

Considering the node representations vary in different layers, Without loss of generality, we replace max i ∈ [ n ] ∥ x i ∥ 2 with max i ∈ [ n ] ,j ∈ [ l ] ∥ ∥ ∥ x j i ∥ ∥ ∥ 2 to find the maximum node representation, instead of node feature. Then recall the PAC risk bound defined in Eq. (29),

We use the sum of both above bounds as the final risk bound for a GCN with KL-divergence loss for the label distribution learning problem.

As [49] suggests that 0 is replaced by a very small value, say γ > 0 , for division by 0 when implementing KL divergence, then for probability distribution p , q ∈ R m with p i ≥ γ, q i ≥ γ

thus there exists a constant b ≥ -ln γ such that KL( · , · ) ≤ b (e.g., b = 35 for γ = 1 × 10 -15 ). Hence, all of the above concludes the proof.

H Supplement F: Additional Theoretical Analysis

H.1 Uniform Distribution Phenomenon

It is observed in the experiment that given a fixed large hidden dimension f , normalized adjacency matrices are more easily to initialize approximate uniform attention than unnormalized adjacency matrices. In the following, we provide a theoretical analysis of the reason behind this phenomenon.

H.1.1 Normalized adjacency matrix are more likely to initialize uniform attention

For simplicity of analysis, we use one node's attentions from a single meta path graph in the study. Same analysis can be applied to all nodes and all meta-path graphs, without loss of generalizability.

Denote A ∈ R n × n the adjacency matrix of the meta-path graph from a meta-path P , and W ∈ R n × f the learnable weight of the meta-path graph ( W is indeed W i 0 in Eq. (1) shown in the main manuscript. We drop W i 0 's subscript 0 and superscript i here for simplicity, as our analysis is based on a single meta-path graph). W [: , i ] ∈ R n × 1 denotes the column vector of W , and A [ i, :] ∈ R 1 × n is a row vector recording node i 's connections ( n is the number of nodes in the meta-path graph).

A uniform weight initialization is with bounds [ -1 √ f , 1 √ f ] for all the learnable matrices. d denotes the average degree of the graph adjacency matrix.

Theorem 11. Denote A and ˜ A the adjacency matrix and normalized adjacency matrix of a graph, respectively. For node i in the graph, its attention is proportional to A [ i, :] · W [: , i ] ∼ O ( d √ f ) for the adjacency matrix, but proportional to ˜ A [ i, :] · W [: , i ] ∼ O ( 1 √ f ) for normalized adjacency matrix.

Proof. Normalization of A (ignoring the self-loop for simplicity) gives:

Denote e j ∈ R 1 × n a one-hot encoded vector with only j th position has value 1 ( e.g. e 2 = (0 , 1 , 0 , 0 , 0) . Because ∥ e i ∥ = 1 , it's easy to know that e j , ∀ j can be used to construct an orthonormal eigenbasis. Denote E i as the index set of edges for node i . For example, if A [ i, :] = (0 , 1 , 1 , 0 , 0) , then E i = { 2 , 3 } , because index position 2 and 3 has 1 ( i.e. edge connections) to node i . Then we can decompose A [ i, :] and W [: , i ] as follows,

̸

where c k is a scalar for basis e k and c k is bounded by [ -1 √ f , 1 √ f ] . Assume that c k = 0 for k ∈ E i (meaning that the scalar is nonzero when nodes i and k are connected), then we have

where d i denotes the degree of node i ( i.e. d i = ∑ A [ i, :] ). Compared to Eq. (42) vs. Eq. (40), it is easy to conclude that f plays more significant role in Eq. (42). Meaning that an unnormalized adjacency matrix affects the final attention with the average degree of the matrix while normalized adjacency matrix is dominated by the square root of hidden dimension f . With a relatively large hidden dimension, it is easy to see that normalized adjacency matrices are more likely to be dominated by the hidden dimension size f and produce uniform distribution of learned attention in the beginning stage.

H.2 Benefits of proposed attention structure in terms of label distribution learning

Similar structure of our proposed attention structure is applied when combining GNN and transformer structure, we provide an analysis on why such proposed attention structure is beneficial to graph label distribution learning. To show this, we first leverage the conclusion from Eq. (34) to obtain a bound of graph label distribution learning on normal l GCN layer.

There are three parts for the bounds, the part O

as it is directly related to maximum degree d of graph adjacency matrix and is related to our change in the GNN structure. It is easy to observe that our proposed attention structure follows the same GCN update except the adjacency matrix is now learnable from features and combined with topology space. So we assume that our model is a dynamic graph version and we focus on how the changes in the learnable graph could affect the bound. We show in the following that our method reduces the maximum degree of graph adjacency matrix and therefore tightens the bound for graph label distribution learning.

Our attention layer learns a feature space topology and intersects its feature neighbor with its topology neighbor to get a consistent neighbor set agreed by both feature and topology.

Theorem 12 (Equal priority of feature and topology space) . Denote original graph structure as A, propagation graph for attention structure as ˜ A , denote δ ( A [ i, :]) as the i th node's degree. ∆ (A) as maximum degree of adjacency matrix A and arg max i δ ( A [ i, :]) as the node index that has the maximum degree:

Proof.

With ⊙ operator, for any node i, it is easy to observe that

Assume without loss of generality that A has ∆ (A) at node j , i.e , arg max i δ ( A [ i, :]) = j . and ˜ A has ∆ (A) at node k,i.e, arg max i δ ( ˜ A [ i, :]) = k . For arbitrary node i in ˜ A ,

We note that the equality sign only appears iff k = j and δ ( ˜ A [ k, :]) = δ ( A [ k, :]) which relies on ˜ A f at row k to be exactly the same as A at row k . The probability is related to the consistency of feature similarity and topology structure.

For a graph with relatively low homophily score p i for a specific node i with p i defined as

where Label ( i ) for a node i is the dominate class of its distribution and Neighbor ( i ) is defined as topology neighbor set of node i . | Neighbor ( i ) | indicates the size of the set.

Assume that feature similarity strongly relates to label connection among nodes,i.e, features are similar between samples if their label distribution are close. The learnt ˜ A f at row k satisfies the equality condition iff p k = 1 . For practical graphs in general, such homophily score rarely exist and therefore our methods should reduce the maximum degree of the propagation graph in most cases.

Table 14: Runtime results for 100 training epochs. The numbers of target nodes are labeled beside their corresponding dataset names.

H.3 Runtime Discussion

To verify the efficiency of our method, Table 14 shows the runtime for 100 training epochs for our baseline methods, Each method has been trained with 100 epochs. We can observe that our proposed HGDL algorithm enjoys good scalability by attaining runtime results that are comparable to the baseline GCNs. Compared to other baseline such as SeHGNN, HGDL is in general much faster. This indicates that our method is more efficient for learning heterogeneous networks. This can be mainly attributed to the automated learning of weights to control individual meta-paths and then integrates them to yield optimal graph topology homogenization for label distribution learning.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our study aims to generalize Label Distribution Learning from IID data to the non-IID, networked data, and we have made this claim clear in the Abstract and Introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discussed the limited performance gain on CAD and CLD metrics in Section 6.3 and provided Runtime analysis in Supplement F Section H.3.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have analyzed the generalization error bound of the proposed algorihtm in a PAC-Bayes regime and provided the complete proof in Supplement D and E.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

[Yes]

Justification: We have provided tuned parameter settings in provided Github link and reported their impact on empirical results in Supplement C.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have uploaded the code and benchmark datasets as the supplementary submission in provided Github link.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have presented the creation and preprocessing steps of the benchmark datasets and the implementation details of our algorithms in Supplementary B and C, respectively.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We generated experimental results from repeated experiments (including different train/test split) and leveraged t-test to validate the statistical significance in settings where our algorithm outperform its competitors. We reported its win/tie/loss in Section 6.2 where 'win' indicates that our algorithm outperforms the compared models with confidence level at a level of 90%.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We disclosed our computing node setup. Note, our algorithm is not computational demanding and could be generalized onto various single-GPU machines.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We revised the COE and all agreed on it.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We demonstrated multiple potential application of our algorithm with the benchmark dataset, including the use case of Urban Functionality Delineation, which has been leveraged as the motivating example of our proposal in Introduction.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: NA

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: NA

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The artifacts generated from this paper will be published under CC-BY 4.0 license for public access and non-commercial use.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: NA

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: NA

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies heterogeneous graph label distribution learning with the aim of predicting label distributions of unlabeled nodes in a heterogeneous graph. This paper elaborates the challenges for generalizing LDL into networked data, and proposes an LDL algorithm HGDL to overcome the challenges. Besides, this paper derives the PAC-Bayes error bound for HGDL and conducts experiments to show the superiority of the proposal.\n\nSTRENGTHS:\nThis paper studies a new problem, i.e., label distribution learning in heterogeneous graphs. Besides, this paper proposes an end-to-end HGDL learning approach to jointly learn the optimal meta-path graph topology and node representation. The effectiveness of the proposed method is studied theoretically and empirically.\n\nWEAKNESSES:\nThe contribution of this paper is unclear. This paper attempts to combine heterogeneous graph learning with label distribution learning. However, the paper addresses challenges about the topology, heterogeneity, and inconsistency in terms of instances, and pays little attention to the challenges of learning label distributions of instances, i.e., the proposed challenges do not arise from label distributions. Therefore, the contributions for LDL are not clear.\n\nQUESTIONS:\n1.\tWhy can't we just replace the output layer of existing heterogeneous node classification model with MSE (Mean Squared Error) or KL loss to learn the label distribution? What challenges arise when replacing categorical labels with label distributions in HGDL? And which of these problems does this paper address? What is the contribution of this paper to the field of label distribution learning?\n2.\tWhy use PAC-Bayes theory to analyze the error bound instead of PAC theory?\n\nLIMITATIONS:\nI don't believe that the paper has a potential negative social impact.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of heterogeneous graph label distribution learning. To deal with it, this paper proposes an HGDL method that optimizes meta-path graph topology and aligns it with nodal features for consistent message-passing, backed by theoretical support. Experimental results on five datasets demonstrate the effectiveness of the proposed model.\n\nSTRENGTHS:\nS1. This paper is the first to investigate the LDL problem in heterogeneous graphs, which seems interesting.\n\nS2. HGDL effectively combines meta-path aggregation and transformer-based methods to ensure consistent node label distribution learning, validated by both theoretical analysis and empirical studies.\n\nS3. The contributions are clearly written, with codes and datasets provided for reproducibility and practical utility.\n\nWEAKNESSES:\nW1. The intuitions behind the techniques lack explanation. More details can be found in Q1 and Q2.\n\nW2. The approach section is somewhat difficult to follow and could benefit from improved presentation.\n\nW3. It would be beneficial to include baselines for the LDL problem, such as GLDL [1]. While these methods are not specifically designed for heterogeneous graphs, it is still important to treat heterogeneous graphs as homogeneous ones and apply these methods for comparison. This will provide a more comprehensive evaluation of the proposed approach.\n\nW4. This paper compares only three baselines, which are relatively dated for heterogeneous graph learning. For example, GCN was published in 2017, HAN in 2019, and SeHGNN in 2023. This makes it challenging to convincingly demonstrate the advantages of the proposed HGDL. Thus, it is essential to include more recent baselines, such as HINormer [2].\n\nReferences:\n\n[1] Y. Jin, R. Gao, Y. He, and X. Zhu, “Gldl: Graph label distribution learning,” in Proceedings of the 38th Annual AAAI Conference on Artificial Intelligence, 2024.\n\n[2] Qiheng Mao, Zemin Liu, Chenghao Liu, and Jianling Sun. \"Hinormer: Representation learning on heterogeneous information networks with graph transformer\", in Proceedings of the ACM Web Conference, 2023.\n\nQUESTIONS:\nQ1. Why only search for meta-paths connecting nodes of the target type rather than aggregating information from neighbors of different types?\n\nQ2. Why can the design in Section 4.2 effectively capture global feature consistencies? Could you provide more detailed explanations?\n\nQ3. Could you provide an explanation for why HGDL performs worse on the CLD metric?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel approach to Label Distribution Learning (LDL) specifically tailored for heterogeneous graphs, addressing the inherent complexities and challenges associated with this domain. By highlighting the necessity of LDL in heterogeneous settings and outlining the unique challenges involved, the paper lays a foundation for advancing research in this emerging field.\nThe proposed method is underpinned by a robust theoretical framework, providing a coherent rationale for its design and implementation. \nExperimental validation across five distinct datasets and evaluation using six metrics demonstrate the method's effectiveness over established baselines.\n\nSTRENGTHS:\n1.This paper is the first to identify the necessity of Label Distribution Learning (LDL) on heterogeneous graphs, addressing the unique challenges and complexities associated with this task. \n2.The proposed method is well-grounded in theory, providing a solid foundation for its design and implementation. The authors present a clear and thorough theoretical framework that supports the efficacy and rationale behind their approach, enhancing the credibility and robustness of the method.\n3.The effectiveness of the proposed method is demonstrated through extensive experiments on five diverse datasets and six different metrics. The results consistently show that the method outperforms the baselines, indicating its superiority and practical applicability across various scenarios and evaluation criteria.\n\nWEAKNESSES:\n1.The paper categorizes current LDL methods into three distinct types in the related work section. However, the experimental evaluation only includes three models (GCN, HAN, and SeHGNN) with the KL-divergence loss function as baselines. This limited selection raises questions about whether the experiments sufficiently demonstrate the proposed method's superiority over the broader range of existing LDL methods.\n2.The study employs γ as a hyperparameter in the proposed model but does not provide any analysis of how variations in this parameter affect the model's performance. A thorough analysis of the hyperparameter's impact would offer valuable insights into the model's sensitivity and robustness.\n3.The paper does not present a detailed comparison of the computational time required for the proposed method versus the baselines. Without this information, it is unclear whether the proposed method is more efficient in terms of runtime.\n4. The motivation of this paper is not very convincing. It seems that this work is done just because LDL has not been applied in heterogeneous graph. It is not clear why LDL can solve some key problems in heterogenous graph.\n\nQUESTIONS:\nI don't have any questions.\n\nLIMITATIONS:\nWhile the authors provide a complexity analysis and mention that scalability is not the main concern of this paper, they acknowledge that the current model may not scale well with larger datasets. Although a modification to improve scalability is suggested, it is left for future work, indicating that the current version might struggle with large-scale data.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper advances Label Distribution Learning (LDL) into the realm of graph domains, specifically addressing the heterogeneous graph label distribution learning (HGDL) problem. The authors highlight that graph heterogeneity, reflected in node types, node attributes, and neighborhood structures, poses significant challenges for generalizing LDL to graphs. To tackle these challenges, the authors propose a new learning framework with two key components: \n1.Proactive Graph Topology Homogenization: This component focuses on learning optimal information aggregation between meta-paths to address node heterogeneity before the embedding learning phase.\n2. Topology and Content Consistency-aware Graph Transformer: This component uses an attention mechanism to learn the consistency between meta-paths and node attributes, ensuring that network topology and nodal attributes are equally emphasized during label distribution learning.\n\nSTRENGTHS:\n1.The introduction of a framework that proactively addresses graph heterogeneity and incorporates consistency-aware mechanisms is a novel contribution to the field.\n\n2.The use of KL-divergence and additional constraints in an end-to-end learning process provides a robust solution for label distribution learning on graphs.\n\n3.The theoretical and experimental validations are thorough, and the availability of code and datasets promotes transparency and reproducibility.\n\nWEAKNESSES:\n1. The paper lacks an analysis of the algorithm's complexity, and the framework diagram in Figure 2 is somewhat cluttered.\n\n2. The authors' motivation is to address the challenges posed by graph heterogeneity, but these challenges are not clearly described in the introduction.\n\n3. For optimization formula (3), the authors need to provide a detailed explanation of the advantages of this design.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}