GR E T O : R EMEDYING DYNAMIC GRAPH TOPOLOGYTASK DISCORDANCE VIA TARGET HOMOPHILY
Anonymous authors
Paper under double-blind review
A BSTRACT
Dynamic graphs are ubiquitous across disciplines where observations usually change over time.
Regressions on dynamic graphs often contribute to diverse critical tasks, such as climate early-warning and trafﬁc controlling.
Existing homophily Graph Neural Networks (GNNs) adopt physical connections or feature similarity as adjacent matrix to perform node-level aggregations.
However, on dynamic graphs with diverse node-wise relations, exploiting a pre-deﬁned ﬁxed topology for message passing inevitably leads to the aggregations of targetdeviated neighbors. We designate such phenomenon as the topology-task discordance, which naturally challenges the homophily assumption.
In this work, we revisit node-wise relationships and explore novel homophily measurements on dynamic graphs with both signs and distances, capturing multiple node-level spatial relations and temporal evolutions.
We discover that advancing homophily aggregations to signed target-oriented message passing can effectively resolve abovementioned discordance and promote the aggregation capacity. Therefore, a novel GReTo is proposed, which performs signed message passing in immediate neighborhood, and exploits both local environments and target awareness to realize high-order message propagation.
Empirically, our solution achieves signiﬁcant improvements against best baselines, notably improving 24.79% on KnowAir and 3.60% on Metr-LA.
I NTRODUCTION
Graph-structured data mining has become a popular technique in numerous disciplines, such as social networks You et al.
(2022), road networks Chen et al.
; Bai et al.
(2020), and molecule analysis Hamilton et al.
; Abu-El-Haija et al.
However, existing solutions to graph mining usually make the assumption of homophily on graphs where connected nodes tend to share similar features or have the same labels (targets).
Actually, in real-world graphs, the homophily assumption does not always hold on Zhu et al.
Thus, Graph Neural Networks (GNNs) considering heterophily are proposed to break the homophily assumption, which disentangle the complex neighborhood components Ma et al.
Du et al.
and model the edge diversity Zhu et al.
(2021a)
; Wang et al.
(2022a) by separately aggregating similar and dissimilar signals Bo et al.
(2021); Yan et al.
Despite achievements, heterophily GNNs are mostly investigated on classiﬁcation tasks over static graphs while less explored on node-level regressions over dynamic graphs.
Therefore, it provides an opportunity to dissect how edge-type disentanglement boosts regression capacity on dynamic graphs.
Regression tasks are more challenging than classiﬁcation as the latter only considers discrete labels with much tolerance Wang et al.
(2022b).
Actually, nodes in dynamic graphs are more prone to suffer complex neighborhood distributions Ma et al.
(2022) with different edge types, potentially incurring misleading message passing when aggregating target-deviated neighbors.
The misleading message passing is formally designated as the topology-task discordance in our work (see Fig. 1(a)).
We take trafﬁc volumes of road networks as an intuitive example of edge diversity in Fig. 1(b)-(c).
The neighboring intersections sharing an upstream-downstream connectivity can be positively correlated, while interactions locating parallelly on the same Origin-Destin transition tend to be negatively correlated with a contended relationship.
These two-type edges respectively account for homophily and heterophily components.
Consequently, uniform aggregations on these two-type neighbors will involve interfered noise and deteriorate the performances of GNNs, as not all of them have consis- tent evolution direction towards targets.
Empirically, in four real-world dynamic graphs, both low homophily ratios within intra-graph frames and across temporal adjacent frames (Fig. 1(d)) imply that physically-connected nodes are not necessarily with close observations 1 or with same variation directions.
This not only supports the argument of topology-task discordance, also manifests the universality of such phenomenon across different dynamic graphs 2 .
Furthermore, due to the heterogeneous local structures and neighborhood distributions, the topology-task discordance can be propagated to high-order neighborhoods.
In other words, the optimal receptive ﬁelds should be adaptively and efﬁciently constructed to realize controllable neighborhood aggregation, thus avoiding noise involvement.
Therefore, for dynamic graphs, remedying topology-task discordance in both immediate and high-order neighborhoods is urgently desired.
[Figure]
Challenges.
However, since observations of dynamic graphs always vary in continuous data space and the node-wise relationships exhibit spatial-temporal property, adapting the conventional homophily theory to address such topology-task discordance is still challenging.
The key obstacles can be summarized as 1) how to determine which pairs of nodes belong to homophily components without categorical labels, 2) how to involve targets to reconstruct node-wise correlations, thus materializing remedied and powerful target-oriented aggregations, 3) how to exploit the dynamic local neighborhood environments to achieve personalized high-order propagations.
Present work. Our work empirically and theoretically elucidates the existence of topology-task discordance and explains the failure of homophily GNNs on dynamic graphs.
To get rid of such dilemma, we propose a novel GNN to Remedy Topology-task discordance via Target-homophily (GReTo).
Firstly, we extend the node-wise relations to a triple tuple based on signed-distance proximity, and construct two measures including intra- and inter-graph homophily to capture diverse spatial-temporal relations and overcome lacking categorical labels.
Secondly, by introducing the target awareness with a transition homophily predictor, we incorporate two signed homophily measures to facilitate target-oriented message passing, renewing the activeness of GNNs.
Finally, instead of imposing a nested or bi-level inefﬁcient optimization Xiao et al.
(2021), we devise an adaptive layer-wise importance measurement to promote the immediate neighborhood aggregation towards high-order propagations, by identifying the informativeness of each propagation step relative to the expected targets.
We evaluate our solution on four dynamic graphs and successfully achieve 3.20% to 24.79% improvements against baselines on MAPE, where KnowAir ( ↑ 24.79%) with higher intragraph negative heterophily ratios (Tab. 3) especially beneﬁts from ﬂexible signed message passing.
Contributions.
(1) We formalize a dynamic graph homophily theory, jointly characterizing multitype node-wise relations considering spatial-temporal property.
(2) On dynamic graphs, we analyze the topology-task discordance and corresponding solution from three aspects, a) the existence, b) the opportunity of boosting aggregation capacity, c) quantitative relationships between neighborhood environments and target-oriented information in each GNN layer.
(3) We propose GReTo, consisting of a signed target-oriented message passing and layer-importance based high-order propagation, to reﬁne the topology adapting to downstream regression tasks.
R ELATED WORK
GNNs have become an admirable tool of diverse graph-structured data mining Song et al.
Kipf & Welling (2016); Abu-El-Haija et al.
To boost the aggregation and representation capacity of traditional GCNs, Graph Attention Network Veliˇckovi´c et al.
(2018) modiﬁes the conventional GNNs by allowing ﬂexible node-level attention with efﬁcient parallel training, while Directional GNN Beaini et al.
(2021) exploits directional derivatives to endow GNNs with the anisotropic property and overcome the oversmoothing issue.
Recently, multiple node-level relations motivate the research trend of homophily theory that measures the consistency between the original topology and node-wise features (or labels)
Zhu et al.
(2021b);
Bo et al.
These techniques respectively design bi-kernel aggregations Du et al.
Yan et al.
and high-order neighborhood modeling Ma et al.
; Zhu et al.
(2020) to advance the standard GNNs towards more powerful aggregation capacity in addressing heterophily.
Besides, researchers carefully devise various high-order propagations by a bi-level learn-to-stop optimization Xiao et al.
(2021) or layer-shared weighted aggregation Chien et al.
Even ﬂourishing, graph theories mostly focus on classiﬁcation over static graphs, thus resulting in the opportunity of adapting emerging theories to dynamic graphs.
Dynamic graph learning usually adopts RNN backbones Ruiz et al.
; You et al.
Bai et al.
(2020) in a spatial-temporal manner, where they construct various adjacent matrices via similarity Zhou et al.
(2020), or learnable embedding product Wu et al.
(2020b).
However, dynamic graphs inherently suffer the edge diversity and dynamics, which are inclined to the topology-task discordance.
Existing dynamic graph learning usually fails to consider such multiple node-wise relations and seldom theoretically justiﬁes the rationality of their adjacence designs.
In contrast, we reﬂect on how the homophily theory on node-wise relations lifts the regressions over dynamic graphs. We further investigate target-oriented directional message passing and personalized aggregations with theoretical guarantees.
Detailed related works can be found in A.3.
P RELIMINARIES
Let T = { 1 , 2 , ..., T } be a temporal step set and denote the dynamic graphs as G =
{G 1 , G 2 , ..., G t , ..., G T } .
Given step t , G t = {V , X t , A t } represents the valid observation of graph
G .
Speciﬁcally, the node set V = { v 1 , v 2 , ..., v N } is with the cardinality of N , and the matrix
A t ∈ R N × N describes the static weighted adjacency relationship of G t .
Let F be the number of feature dimensions of each node 3 , then X t = { x t 1 , x t 2 , ..., x tN } ∈ R N × F serves as the feature matrix.
The goal of our dynamic graph regression is to derive an optimized graph learning function F ∗ g to predict the observations across the whole graph for the next step, i.e., (cid:98) X T +
=
F ∗ g ( G , X t , A t | t = 1 , 2 , ..., T ; Θ ) .
We name { ( X t , A t ) | t
= 1 , 2 , ..., T } and X T +1 respectively as the historical and targeted observations, while Θ refers to all learnable parameters.
3. D YNAMIC GRAPH HOMOPHILY THEORY
We ﬁrst introduce the conventional graph homophily ratio in classiﬁcation Zhu et al.
; Ma et al.
(2021), which is deﬁned as the fractions of nodes in their neighborhoods that with the same label y i as v i , i,e., ho
[Equation]
Here, N ( v i ) refers to the neighboring node set of v i .
In contrast, we propose our dynamic graph homophily theory to adapt the continuous data space.
Our dynamic graph homophily theory includes descriptors of homophily measurements and local neighborhood environments, which are two factors of node-level and neighborhood-level statistics that inherently affect the information propagation.
Concretely, we ﬁrst exploit signed distances to extend the binary relations to a triple tuple, compensating for non-explicit class boundaries.
For dynamic graphs, we formulate an intra-graph spatial homophily to capture node correlations in same graph frames and an inter-graph transition homophily to extract temporal evolution between adjacent temporal frames.
Second, to numerically depict the neighborhood on both structures and neighbor compositions, we integrate the topological degrees with homophily distribution as local neighborhood environments.
Deﬁnition 1 (Intra-graph spatial homophily and Inter-graph transition homophily)
Both intraand inter-graph homophily can be categorized into three classes according to the feature distances by a proximity threshold ε , with the signs indicating the direction of proximity.
Given a serial graphs G , temporal step t and node v i , the intra-graph homophily ( π sij ) t and inter-graph homophily ( π Tij ) t are respectively deﬁned by a symmetric formation (see Figure 6(a)) as,
[Equation]
Deﬁnition 2 (Local neighborhood environments (LNE))
LNE consists of local structure descriptions and neighborhood distributions.
Given node v i , we let d i describe the local structure and further partition the nodes of its neighborhood N ( v i ) into three groups based on node-wise feature distances.
In detail, the neighboring components those are proximal, positively deviated, and negatively deviated away the central node v i are respectively considered as spatial homophily ( N ( v i ))
s , positive heterophily ( N ( v i ))
qp , and negatively heterophily ( N ( v i ))
qn .
We can transfer the neighborhood components into a numerical Neighborhood Distributions vector π v i ,
[Equation]
2) where ( p s , q p , q n ) = { p ( π sij = ε ) , p ( π sij = 1) , p ( π sij = − 1) } .
Then the local neighborhood environment can be denoted as LNE v i =
( d i , p i , q p
i , q n i ) .
D ISCORDANCE BETWEEN TOPOLOGY AND TASKS
In this section, we show the discordance between the topology and task, when adopting traditional homophily GNNs.
Given a serial graphs G and node v i , (1) the ratio of intra-graph spatial homophily π v i ( p s ) is simpliﬁed to p i , while the ratio of other node-level relations is denoted as q i = 1 − p i , (2) the target coefﬁcient γ i quantiﬁes the relationship between the targeted step T +1 and a given step t by x T
+1 i
= γ
i x ti .
Our analysis only considers aggregations from frame T to T + 1 , and let x T +1 i
= γ
i x Ti .
We’ll ignore the temporal superscript for simpliﬁcation in the following analysis.
Actually, given a target coefﬁcient γ i , a successful aggregation can be equivalent to ﬁnding a valid d i to construct an appropriate neighborhood for achieving targets.
Therefore, we establish the relationship between γ i and degree d i , to ﬁnd what conditions will the homophily-assumed graph aggregation fail.
Theorem 1 (Proof in Appendix B.1.)
Consider impose a homophily GNN F g on X for node-level regres- sions, the relationship between the expected degree d i and the target coefﬁcient γ i is,
[Equation]
Theorem 1 demonstrates that homophily aggregations cannot ﬁnd the optimal neighborhoods to achieve the targeted results when γ i < p i and γ i > 1 .
In other words, the homophily GNNs will fail when the gap between historical observations and targeted prediction value is large.
R EMEDY TOPOLOGY - TASK DISCORDANCE ON DYNAMIC GRAPHS
In this section, we present our GReTo, which consists of two stages, Signed Target-oriented Message
Passing, and Personalized high-order Layer Propagation.
5. S IGNED T ARGET - ORIENTED M ESSAGE P ASSING
The following Theorem states that given an extremely large or small target coefﬁcient, advancing to signed aggregations can renew the activeness of GNNs.
Theorem 2 (Proof in Appendix B.2.)
Given a central node v i , and target coefﬁcient γ i < p i or γ i > 1 , the aggregation capability can be improved by a sign-preserved directional message passing that separately aggregates positive and negative heterophily compositions.
Under such message passing, we denote s p >and s n < 0 as the signed kernels, then the aggregation algorithm can conditionally converge to x T
+1 i where d i satisﬁes the following two conditions,
[Equation]
where the expectation of neighbors with intra-graph homophily denotes d i p i (1 ± ε ) x
i .
N + ( v i ) and N − ( v i ) denote the positively and negatively heterophily neighborhoods, and w p , w q are learnable weights.
Detailed conditions for obtaining targets are demonstrated in Appendix B.2.
With these derived conditions, we can estimate whether the GNN can successfully aggregate the neighbors towards targets when the dataset is given.
Therefore, the key to our target-oriented message passing becomes investigating both intra- and inter-graph homophily to discover the target-beneﬁcial neighborhoods, and devising meaningful directional aggregation.
The technical stages are three-fold, 1) discover the target-oriented homophily and heterophily neighborhoods, 2) predict node-wise inter-graph transition homophily to enable target awareness, and 3) adaptively highlight different types of neighborhoods for signed target-aware aggregations.
The three components are elaborated as follows.
Target-homophily neighborhood discovery.
To explicitly distinguish target-oriented neighboring nodes, we resort to our dynamic homophily theory with both signed direction and proximity.
Considering that the signs in these two homophily measurements indicate the directional consistency, we couple the spatial one and transition one to derive consistency between the intra-graph connections and the targeted graph.
The signed target-oriented neighborhood disentanglement matrix (cid:99) M can be achieved by an element-wise division ∅ , i.e., (cid:99) M ij = (cid:99) π T ij ∅ π sij , where (cid:99) π T ij is the estimated transition homophily that will be predicted in the next subsection.
This design of division enables each (cid:99) M ij ∈ { 1 , − 1 , ε, − ε, 1 ε , − 1 ε } in (cid:99) M to possess informative edge descriptions, i.e., each edge value can not only discriminate the target-oriented nodes (with positive signs) as good neighbors, but also measure the degree of node-wise proximity where larger values imply more target-oriented consistency and contribution.
By splitting the (cid:99) M based on the edge signs, we can achieve the targetoriented homophily and heterophily neighbors by,
[Equation]
In Eq 5, [ (cid:99) M ] P is the disentangled aggregation matrix with positive elements interpreting the targethomophily adjacent connections, while [ (cid:99) M ] N composes negative values explaining the targetheterophily relationship.
Noted that although the original triple relations are shrunk into the signed matrix for efﬁcient implementations, the directional proximity indicating the informativeness for aggregation towards targets is carefully preserved by diverse values in (cid:99) M .
Inter-graph transition homophily predictor.
Since the transition homophily is not available during inference stages, we formulate a sequence classiﬁcation task to predict which type of relation that the transition homophily belongs to.
However, computations among node-level transition homophily are inefﬁcient, we thus reduce the transition homophily to a self-transition homophily between adjacent steps Sig ti =
( π Tii ) | t − 1 → t , which can be available from historical observations.
Then, we leverage the nice property of periodicity and continuity in time-series to construct the sequence learning.
[Figure]
This time-series learner concatenates two parallel LSTMs and one 1D-convolution (Conv1D) to capture the evolution and trend dependence by taking stepwise historical node observations and self-transition homophily [ X t ; Sig t ]( t = 1 , 2 , .., T ) as inputs.
Thus, we obtain the estimated next-step representation by,
[Equation]
We then utilize the time-series representation (cid:100)
Sig T +1 R for a three-category classiﬁcation where the class probability is estimated with an MLP parameterized by W s , i.e.,
[Equation]
The estimated (cid:98) p , (cid:98) q p , (cid:98) q n refer to probabilities of three homophily categories that (cid:100)
[-+
-]Sig T +1 R belongs to, and then we can obtain the ﬁnal (cid:100) Sig T +1 based on these probabilities.
Disentangled signed directional message passing.
With modiﬁed transition homophily Sig T +1 , the disentangled target-oriented matrix can be altered as (cid:99) M ij = (cid:100)
Sig ti ∅ ( π sij ) t .
To accommodate separate aggregations over different types of node-wise connections, we take advantage of the positive target-homophily matrix [ (cid:99) M ] P and negative target-heterophily matrix [ (cid:99) M ] N to respectively ﬁlter the adjacence (cid:98) A i where we impose a self-loop on the adjacence (cid:98)
A =
A + I .
Thus, the two aggregation kernels can be realized as,
[Equation]
where (cid:12) is an element-wise Hadamard Product for kernel ﬁltering. f L and f H are respectively interpreted as a low-pass ﬁlter and a high-pass ﬁlter in graph convolutions.
Then the disentangled signed directional message passing can be formulated as,
[Equation]
where N L ( v i ) and N H ( v i ) denote the target-homophily and target-heterophily neighbor set regarding v i , and α adjusts the proportion between two compositions.
In addition, w p ∈ W P and w n ∈ W N are two learnable parameters for feature transformation.
In Eq.
9, the negative kernels can make sense when the aggregated neighbors are exactly negatively correlated with targets, which potentially closes the gap between historical node observations and targeted ones. Thus, our strategy can explicitly model multi-type edges and boost our aggregation power.
5. P ERSONALIZED HIGH - ORDER LAYER PROPAGATION
Since the propagation step is a discrete value, it is intractable to directly optimize such discrete values.
Instead of a bi-level inefﬁcient optimization Xiao et al.
(2021), we exploit the local neighborhood environments to quantify the informativeness of node representations at each propagation step, which determining when to stop the propagation in a soft manner.
Concretely, we make the following observations.
Relationship between local neighborhood environments and propagation steps.
Given node v i , (1) the optimized propagation step is determined by the proportion of target-homophily neighborhoods (cid:101) p i (rather than intra-graph homophily p i ) and the substructure description d i .
(2) the smaller values of d i and (cid:101) p i are, the larger propagation step K is expected.
By denoting INFO ( h ki ) as the target-oriented information encapsulated in the representation of the k -th propagation step in message passing, Theorem 3 demonstrates the quantitative relationship between the informativeness of each propagation layer and the local neighborhood environments, to verify our qualitative analysis.
Theorem 3 (Proof in Appendix B.3) Consider a series of dynamic graphs G that conform to the neighborhood distribution N ( v i )
∼ π v i ( p, q p , q n ) where we enforce q n = ηq p , p + q p
+ ηq p = 1 .
Consider x T +1 i
= γ
i x Ti > x Ti with γ i > 1 , the quantitative relationship among the layer-speciﬁc target-oriented information INFO ( h ki ) , substructure d i , and the target-homophily (cid:101) p i is as follows.
Under this case, the expected target-homophily compositions of neighborhoods (cid:101) p i becomes q n , and the difference between the probability of hitting beneﬁcial nodes in the second layer and the ﬁrst layer ∆ P ( N + ) | 1 − 2 is ( − 1 − 2 η ) q 2 p + q p .
We have,
[Equation]
The symmetric results can be achieved when we have a small γ i [-where x T +
-]satisfying γ i < 1 .
Theorem 3 veriﬁes that in dynamic graphs with complex neighborhood distributions, lower nodelevel activeness and target-oriented homophily must result in larger receptive ﬁelds and vice versa.
Therefore, we are expected to pick the high-order neighbors that are target-homophily.
Adaptive layer importance measurement.
To pick the high-order target-homophily neighbors, we transfer this challenge to measuring the consistency between high-order neighborhood distributions and transition homophily distribution of targets.
First, we exploit the edge composition π v i to construct a neighborhood distribution matrix ND h = NeighH ( X ) =
[ π v 1 ; π v 2 ; ..., π v N ] ∈ R N ×where NeighH ( X ) is a neighborhood disentanglement function on graph-wide features X .
Then ND in the k -th propagation step is ND k h =
NeighH
( A k X ) .
Second, since dynamic graphs experience continuous temporal evolution, the variations of consecutive steps can imply ﬁne-grained ﬂuctuations that progressively approach targets.
Therefore, for each t , we exploit the estimated transition probability distribution on three categories predicted by Eq.
15 to construct the step-wise transition distribution matrix, i.e., ( ND T ) ti =
[ (cid:98) p , (cid:98) q p , (cid:98) q n ] t
∈ R N × 3 .
As a result, ( ND T ) ti naturally encapsulates the target information.
Then, we can measure the consistency between the expected k -th order neighborhood and the target-oriented transition homophily probability by imposing a Hadamard Product to ( ND kh ) t i and ( ND T ) ti .
Then a MAX operation is applied in each row to preserve the maximal result and suppress the non-target-oriented noise.
Besides, a smaller degree tends to encourage aggregating nodes of farther horizons while larger propagation layer should be less weighted.
Thus, the layer importance measurement φ ki for node v i at layer k can be computed by,
[Equation]
where d ki denotes the averaged degree of nodes in v i ’s k -th order neighborhood.
Learn to propagate over high-order neighbors.
The core technique of high-order layer propagation is learning a set of interpretable node-speciﬁc soft weights for aggregating multi-order neighbors.
To achieve such well-normalized aggregation weights, we impose Softmax on the estimated layer importance φ ki , associated with an MLP parameterized by weight W lim to enable it learnable, i.e., (cid:102)
φ ki =
MLP( LiM (( ND kh ) i , ( ND T ) i ) , W lim ) . Given maximal propagation step K , we can obtain the soft weights over layer-wise aggregations ψ[-∈ R N × K by ψ i = Softmax (cid:104)(cid:101) φ 
-]
[Equation]
K by ψ
[Equation]
Softmax
[Equation]
where ψ i is the i -th row in ψ .
So far, we can build our high-order message propagation.
As target-heterophily compositions are less important during aggregations, we only consider the high-order propagation over target-homophily neighborhoods while imposing one-layer aggregation on target-heterophily components.
We have the personalized high-order propagation to achieve K -th layer representation H ( K ) ,
[Equation]
There are two distinctions of our designs.
First, instead of tuning parameters, we instantiate an (cid:126)α ∈ R N as a learnable gate to reduce the ﬁne-tuning burden, i.e., (cid:126)α = σ (MLP( H ( K ) ; W mlp )) and adjust the compositions between target-homophily and target-heterophily.
W mlp ∈ R 2 N ×
N
are learnable parameters.
Second, rather than a complicated routing mechanism, we exploit a learnable but interpretable matrix to reweight the importance of each layer speciﬁc to nodes, contributing to efﬁcient high-order message passing.
We plug our layer propagation into every temporal step.
5. T EMPORAL INCEPTION LAYER AND OPTIMIZATION
Temporal inception layer.
To efﬁciently capture the temporal evolution, we adopt a fullyconvolution based sandwich structure as the temporal inception layer, which is inherited from
STGCN Yu et al.
We can formulate the temporal inception layer as H ( K ) =
Γ ∗ κ ( F ∗ g ( (cid:102) X )) and denote the ﬁnal outputs as (cid:98) y i ∈ (cid:98) Y .
Details can be found in Appendix A.1.3.
Training objective.
The optimization objective of GReTo composes of two parts, the transition homophily estimation implemented by a cross-entropy-based multi-class classiﬁcation, and the dynamic graph regression based on MSE.
The ﬁnal objective can be written as,
[Equation]
E XPERIMENT 6. I MPLEMENTATION DETAILS
Dataset description.
We collect four cross-domain dynamic graph datasets.
All of them possess the mixed property of homophily and heterophily.
Trafﬁc: (1) Metr-LA : Highway trafﬁc status consisting of 207 loop detectors of Los Angeles Li et al.
(2) PeMS-Bay :
Trafﬁc statuses collected by California Transportation Agency, including 325 sensors in Bay Area Li et al.
Climate: (3) KnowAir : PM 2 .
5 Concentrations, covering 184 main cities in China Wang et al.
[Table]
(4) Temperature : Urban Temperatures of the same 184 cities as KnowAir Wang et al.
We construct the graphs based on geographical distances where the node values and virtual topology can be varied across temporal steps.
The parameter δ % , controlling how much percent of nodes are connected with edges, is set as 5% across all datasets.
The statistics and numerical homophily of these datasets are ﬁgured in Table 2 and 3.
Protocols.
We evaluate GReTo on the dynamic graph regression tasks and compare it with stateof-the-art graph architectures.
Training.
We alternately train our model, i.e., train the transition homophily predictor by freezing other weights while utilizing the well-learned transition homophily predictor to continue training the remaining architecture, where Adam SGD Kingma (2014) is the optimization strategy.
We initialize the learning rate of 1e-3 with a weight decay 0.99.
Evaluations.
We employ classic GNNs, topology-reﬁned GNNs, heterophily GNN and spatiotemporal networks for comparisons, and adopt MAPE/RMSE to jointly evaluate the regression performances.
Parameters.
The homophily criteria ε are set as 0.08, 0.10, 0.10 and 0.12, while the maximal propagation steps K are set as 6, 6, 3, 4 on MetrLA, PeMS-Bay, KnowAir and Temperature, according to empirical evaluations.
The size of TCN kernels is set to 1 ∗ 3 on all datasets.
For fairness, for each compared GNN, the number of hidden layers is set to 6 and the hidden dimension for each GCN is set to 64.
For GAT, the number of heads is set to 8 according to the default setting in their papers.
[Figure]
Baseline.
We compare our solutions with respective four types of baselines.
[1] Classic GNN models: (1) GCN Kipf & Welling (2016), (2) GAT Veliˇckovi´c et al.
(2018), (3) GraphSAGE Hamilton et al. (2017).
[2] Topology reﬁned models: (4) SuperGAT Kim & Oh (2020).
(5) EGConv Tailor et al.
[3] Heterophily GNN: H 2 GCN Zhu et al.
[4] Spatiotemporal graph learning: (1) STGCN Yu et al.
(2) MTGNN Wu et al.
(2020b).
(3) GraphWaveNet (GWN)
Wu et al.
(4) DCRNN Li et al.
(5) ASTGNN Guo et al.
More details of baselines can be found in Appendix A.4.2.
Accuracy of transition homophily prediction.
The accurate prediction of transition homophily is the guarantee of the architecture.
Herein, we list the Classiﬁcation Accuracy between the estimated ones and the groundtruth, i.e., 0.78/0.60/0.86/0.68 on respectively four real-world datasets (1)-(4).
As observed, the higher intra-graph homophily tends to lead the higher prediction accuracy and we can also obtain a barely satisfactory result on PeMS and Temperature, which probably beneﬁts from the data property of proximity and continuity.
These promising accuracy can imply the feasibility of our transition homophily predictor and ensure the rationality of following implementations.
Performance comparison.
Comparison results are ﬁgured in Table 1.
The best result is in bold and the second best is underlined.
Our solution almost beats all SOTA baselines, achieving promising performances on four datasets with both 2 metrics.
Promisingly, GReTo improves MAPE by 24.79% and 3.61% respectively on KnowAir and MetrLA, where KnowAir with high negative heterophily q n beneﬁts from ﬂexible signed message passing instead of only similarity-based aggregation.
In detail, classic GNNs fail to involve the transition-oriented temporal information approaching real targets, reasonably resulting in large performance margins between theirs and our GReTo, also inferior to other spatiotemporal networks.
Generally, EGConv and H2GCN obtain better results and we can observe that the higher heterophily of the graphs is, the more superior performances of GReTo,
EGConv and H2GCN reveal.
The underlying distinctions lie in the spatially-varying adaptive ﬁlters of EGConv and heterophily tolerance of H2GCN, supporting our idea of target-homophily topology reﬁnement in dynamic graphs.
Extensive comparisons illustrate that our work advances the interpretation and designs of graph-based theory forward spatiotemporal learning and beneﬁts learning on graphs that are with dynamic topology and heterophily compositions.
[Figure]
Ablation study.
(1) Ablate the target-oriented neighborhood calibration, reducing to traditional
GCN (V-TO).
(2) Replace the high-order layer propagation with layer-wise weighted fusion, reducing to Mixhop (V-Mix).
(3) Reduce the highorder layer propagation to a uniform multi-layer fusion with the same weight (V-ULW).
We report MAPE on four datasets in Table 4.
Overall, promising results conﬁrm the effectiveness of the two well-designed modules.
In particular, target-oriented designs play signiﬁcant roles across all datasets with the improvement ranging from 37% to 61%, while the personalized high-order propagation reveals prominent improvements on Temperature.
Regarding the fusion strategy, the layerwise weighted fusion and uniform aggregation share similar performances while our node-speciﬁc personalized propagation shows a remarkable promotion, demonstrating the ﬁne-grained and interpretable weights efﬁciently capture sufﬁcient information.
Generalization on different γ and multi-step prediction.
We respectively select two datasets for a detailed study. First, we derive the statistical MAPE at different γ ranging from 0.5 to 2.1.
Results are shown in Figure 3(a).
The performances reveal a ’V’ shape with γ increasing, indicating better results when γ approaches 1 while inferior performances as it deviates away from 1.
Similarly but promisingly, our solution exhibits lower errors and more robustness, which can be attributed to signed target-oriented message passing.
Second, we present our multi-step performances against several representative baselines by modifying our framework to a 12-step forecasting with a sequential output where results are in Figure 3(b).
We observe that longer horizons enforce the targetoriented design to gradually lose ground as our GReTo does not tailor for multi-horizon settings.
Even so, our solution still achieves comparable performances with best baselines, which veriﬁes the generalization and potential of GReTo in modeling longer horizons.
This motivates better designs to accommodate both target-oriented and multi-step challenges.
More results on model interpretability and parameter analysis are in Appendix A.4.
C ONCLUSION
In this paper, we propose a dynamic homophily theory by revisiting node-wise relationships from spatial-temporal perspectives and show the opportunity to renew aggregation capacity of conventional GNNs.
Technically, a novel GNN, i.e., GReTo is proposed, which integrates the signed targetoriented message passing and the layer importance based propagation, to reﬁne the topology in both immediate and high-order neighborhoods.
Empirical experiments have validated the superiority of our GReTo on four dynamic graphs.
We will continue to work on how homophily theory improves generalized dynamic graph learning, e.g., edge-type prediction and multi-step series predictions.
M ORE TECHNICAL DETAILS ABOUT PROPOSED GR E T O A.1. F RAMEWORK OVERVIEW
We propose a novel GNN architecture to Remedy Topology via Target-oriented homophily (GReTo), which simultaneously performs adaptive immediate neighborhood aggregation and high-order layer propagation, from the perspective of target-oriented homophily disentanglement.
Speciﬁcally, the proposed two homophily measurements respectively capture node-level intra-graph spatial proximity and inter-graph connections between adjacent steps.
Therefore, we take advantage of both two homophily measurements to ﬁnd target-oriented neighbors, thus calibrating the original topology.
Since GNNs have two major processes, i.e., node-wise aggregations and layer-wise propagation, our GReTo on topology-task remediation correspondingly consists of two stages, Singed Target-oriented
Message Passing, and Personalized high-order Layer Propagation, as illustrated in Figure 5.
We also detailedly demonstrate the spatial and transition homophily as well as the step-wise topology reﬁnement in Figure 6.
[Figure]
c [Figure] l u t i
[Figure]
T e m po r a l c on v o l u t i on
[Figure]
[Figure]
Personalized
Aggregation
[Figure]
Temporal inception
Temporal inception …
Output
[Figure]
Schematic demonstration of dynamic graph homophily
[Figure]
A.1. T ECHNICAL DETAILS ON TRANSITION HOMOPHILY PREDICTOR Motivations.
Although we have derived the formation of target-homophily neighborhood, the transition homophily from step T to T + 1 cannot be available.
However, realizing a predictor to estimate the node-wise transition homophily is intractable due to following two challenges.
1) the unavailability of x T +1 and 2) computing transition homophily between all node pairs is computationally inefﬁcient.
[Figure]
To this end, we make one simpliﬁcation and convert the homophily estimation into an efﬁcient sequence classiﬁcation task to address above issues.
First, we reduce the transition homophily to a self-transition homophily Sig ti .
Our Sig ti =
( π Tii ) | t − 1 → t measures the degree of node-level self variations between adjacent steps, which can be available from historical observations and encourages efﬁcient computations.
Second, to complement the unavailable x T +1 , we ﬁnd the serial inter-graph transition homophily obeys the favorable property of time-series, i.e., the periodicity and continuity.
Therefore, we formulate a time-series classiﬁcation task.
Techniques.
Concretely, the time-series learner concatenates two parallel LSTMs and one 1Dconvolution (Conv1D) to capture the evolution and trend dependence by taking all available historical observations of both node values and self-transition homophily [ X t ; Sig t ]( t = 1 , 2 , .., T ) as inputs.
Thus, we obtain the estimated next-step representation by,
[Equation]
We then exploit the learned time-series representation (cid:100)
Sig T +1 R for a three-category classiﬁcation where the class probability is estimated with an MLP parameterized by W s , i.e.,
[Equation]
The estimated (cid:98) p , (cid:98) q p , (cid:98) q n refer to three probabilities
that (cid:100) Sig T
+
R belongs to corresponding categories, and then we can obtain the ﬁnal (cid:100) Sig T +1 based on these probabilities.
A.1. T ECHNICAL DETAILS ON TEMPORAL INCEPTION LAYER
The efﬁcient fully-convolution based sandwich structure for temporal inception follows the learning order of ’temporal-spatial-temporal’.
Speciﬁcally, the temporal convolution kernel Γ is implemented as a gate by taking self observations as the feature inputs to control the message passing.
By involving both upper and bottom temporal blocks with our GReTo, the whole learning process can be written as,
[Equation]
[Equation] Then the convolution is implemented by the element-wise Hadamard product (cid:12) between selfobservation and the gate, and κ is the length of convolution kernel for temporal aggregation following the setting of Yu et al.
With this well-designed structure, our solution can not only sufﬁciently learn the temporal dependence, but also avoid the insufﬁciency of RNN-based methods.
A. C OMPLEXITY ANALYSIS
As model complexity also plays important role in both training and inference stages, we here present the analysis of additional computation to existing GNNs.
For one step t ∈ |T | , the major computation costs lie in two aspects, (1) in the data-preparation stage, spatial homophily and transition homophily should be pre-calculated, which leads to O ( N ∗ N ) , (2) in the training and inference stages, real-time spatial homophily computation, a binary judgement for matrix sign disentanglement, additional K -times computation for high-order neighborhood distribution as well as matrix product, ﬁnally the MLP for LiM, which lead to O (2 ∗ K + 1 + 1) =
O (2 K + 2) times atomic operation (element-wise matrix product or addition).
Since the data-preparation stage does not occupy training resources and can be computed only once while the computation during training and inference is linearly increasing with maximal layer number K , the additional computation is tolerable for the whole architecture.
A. R ELATED WORK
In this section, we revisit a series of related works on Graph Neural Networks (GNNs), homophily theory in GNNs as well as dynamic graph learning.
A.3. G RAPH N EURAL N ETWORKS (GNN S )
Graph-structured data has received signiﬁcant interest to energize substantial domains including molecular modeling Rong et al.
Beaini et al.
(2021), social networks Zhao et al.
and smart grids Lin et al.
Early GNNs, which categorized to spectral methods, extend the convolution from grid data to irregular graph data by imposing Fourier transformations on graphs and extract informative structures Defferrard et al.
; Xu et al.
After that, researchers ﬁnd that spectral GNNs suffer the issues of high computational costs and non-alignment to vertex domain.
Therefore, spatial methods are proposed to relax the equal number neighbors to ﬂexible neighborhood constructions by directly deﬁning the kernel functions Hamilton et al.
; Kipf &
Welling (2016); Niepert et al.
To enable a global node-wise correlation learning, attention methods of Graph Attention Networks Veli ˇ ckovi ´ c et al.
and ASTGNN Guo et al.
(2019) are subsequently proposed.
However, graph-structured data in real scenarios are more complex than assumptions in most existing research.
For example, multi-type edges and multi-type nodes in graphs are pervasive in many areas such as chemical bonds in molecules and heterogeneous relationships in social networks or citation networks Wang et al.
(2022a).
Also, with the increasing layers of deep graph neural networks, node representations tend to go towards similar by iterative aggregations.
To address these challenges, heterogeneous GNN Zhang et al.
, adaptive relational graph learning Wang et al.
(2022a)
; Schlichtkrull et al.
(2018), and heterophily GNNs Yan et al.
Ma et al.
(2021) have raised.
Even so, adapting the GNN architecture to more complex scenarios still remains as emerging research topic and less explored.
A.3. H OMOPHILY AND HETEROPHILY IN GNN S
The homophily-heterophily theory aims to quantify local neighborhood distributions, i.e., the proportions of different types of neighbors that approaching and deviating from targets.
Given the ’opposites attract’ phenomenon in real-world graphs, emerging GNNs are proposed to tackle the heterophily issue.
Bo et al.
ﬁrst explain multi-type node-wise correlations with low and high frequency components and design frequency adaptation GCN to allow both high and low frequency message passing.
After that, Yan et, al.
propose a signed message passing and degree-correction Yan et al.
(2021), to enable actively identifying different components on graphs by extracting signs of node-level pair-wise similarity, and re-weighting layer-wise information by degree corrections.
At the same time, Zhu et, al.
design a compatibility matrix to model the node heterophily Zhu et al.
(2021b)
and investigate three principles considering neighboring aggregation and high-order propagations, to adapt heterophily scenarios Zhu et al.
Actually, the homophily ratios of different substructures can experience large variances Du et al.
In this way, GPR-GNN Chien et al.
and MixHop Abu-El-Haija et al.
(2019) enable learnable weights over different layers to achieve high-order propagation.
Unfortunately, these high-order propagations cannot specify the weights over node-level, and all above solutions are tailored for classiﬁcation.
A.3. L EARNING ON DYNAMIC GRAPHS
Majority of dynamic graph learning can be classiﬁed into link prediction and node-level regressions.
Tasks of link predictions unversally exist in recommendation system Xia et al.
G¨opfert et al.
(2022), social networks You et al.
and
heterogeneous academic networks Kang et al.
(2022) where these studies exploit the dynamics of user preference and interests to achieve time-varying embedding.
This research line pays more attention to individual embedding rather than the holistic graph.
Node-level regressions such as trafﬁc forecasting Yu et al.
; Guo et al.
; Miao
[Table] [Table]
et al.
and smart grid predictions Lin et al.
; Chen et al.
(2021b), aim to aggregate relevant nodes for approaching future observations.
To capture the dynamic topology, recent works devise various adaptive topology learning strategies, including dynamic similarity Yan et al.
Zhou et al.
; Bai et al.
and attention mechanism Guo et al.
, to quantify weights of different nodes.
Actually, dynamic graph learning tends to be more challenging than static graphs, as these node values are often time-varying and naturally induce the edge diversity over time Chen et al.
Yan et al.
In addition, the imbalanced local structures over nodes Chen et al.
(2021a)
; Wu et al.
(2021) further pose challenges to high-order propagation and topology reﬁnement.
Although there are studies respectively focus on dynamic relation embedding Deng et al.
(2020), and personalized receptive ﬁeld construction Xiao et al.
(2021), none of them investigate multi-relations on regression tasks and consider the potential topology-task discordance.
A. D ETAILS OF EXPERIMENTS A.4. S TATISTICS OF DATASETS
As this work studies the node-wise proximity over dynamic graphs and investigates how the proximity of both homophily and heterophily enhances dynamic graph regressions.
We thus present the statistics of node numbers, total temporal steps and intra-graph homophily of each real-world dynamic graph in Table 2.
Also, the numerical neighborhood distributions of each node are averaged into p s , q p and q n regarding both two types of homophily, i.e., intra- and inter-graph homophily in
Table 3.
Noted that the reported numerical homophily is derived with the well-set homophily criteria ε , i.e., 0.08, 0.1, 0.1 and 0.12 corresponding to four dynamic graph datasets.
Due to the inheret dynamics in these graphs, we can observe that these graphs suffer serious heterophily issue and the inter-graph homophily tends to be higher than that of intra-graph.
We especially observe that datasets of KnowAir and Temperature are with higher negative heterophily q n , and experimental results in Table 1 also demonstrate a more signiﬁcant improvement on KnowAir and Temperature.
These statistics can verify the intuition that modeling homophily and heterophily can disentangle the target-oriented homophily and boost both aggregation and representation capacity in GNNs.
A.4. D ETAILS OF COMPARED BASELINE MODELS
Classic GNNs: (1) GCN: The classical graph convolution neural network Kipf & Welling (2016).
GAT: An attention-based graph method capturing global dependence on the whole graph Veliˇckovi´c et al.
(3) GraphSAGE: A representative spatial method of GNNs to realize graph-based in- ductive learning Hamilton et al.
Topology-reﬁned models: (4) SuperGAT: A self-supervised
GAT designed for noisy graphs, which learns expressive attention in distinguishing mislinked neighbors Kim & Oh (2020).
(5) EGConv: It is a lightweight GCN that designs spatially-varying adaptive ﬁlters Tailor et al.
Heterophily GNN: (6) H 2 GCN : It is a framework for node-level classiﬁcation on graph with homophily and heterophily components by three effective designs Zhu et al.
Spatiotemporal Networks: (1) STGCN: A graph-based spatiotemporal framework with a sandwich structure of temporal-spatial-temporal learning architecture Yu et al.
(2) MTGNN:
Multivariate time-series forecasting model which is capable of adaptively learning uni-directed relations among variables with a graph learning module Wu et al.
(2020b).
(3) GraphWaveNet (GWN):
A state-of-the-art graph-based trafﬁc prediction model that integrates TCNs and GCNs Wu et al.
(4) DCRNN:
A diffusion convolutional recurrent neural network, which combines diffusion graph convolutions with RNNs Li et al.
(5) ASTGNN: An attention-based spatiotemporal network for capturing dynamic ST correlations Guo et al.
[Figure]
A.4. I NTERPRETABLE INTERMEDIATE RESULTS .
To present an intuitive understanding on aggregation process of our GReTo, Fig 9 demonstrates a case study of our GReTo on Metr-LA.
Some important central nodes associated with their neighborhoods and the intermediate results of layer-wise importance are visualized.
It is observed that nodes in graph usually reveal distributed centers and different centers also tend to be connected.
For layerwise importance, it shows that nodes with lower degrees (e.g., Node 7 and 107) tend to propagate farther than nodes with higher degrees (e.g., Node 28 and 78) where the learnable results can verify the intuition of our layer propagation principle and the effectiveness of our GReTo for high-order propagations.
A.4. P ARAMETER SENSITIVITY ANALYSIS
We investigate the three hyperparameters in GReTo to test the model parameter sensitivity.
We illustrate the optimal parameter searching process on all four datasets in Figure 8.
We have the following three observations.
First, we must strike a balance on the edge connections between the lower graph connectivity and reduced graph topology, corresponding to lower and higher connectivity of δ and ε .
Second, it can observed that the larger-scale of graphs tend to require a larger propagation order K , indicating more neurons can accommodate better ﬁtting capacity towards to adapt more intensive data.
Third, although different graphs have their intrinsic property and corrspond to their optimal pa-
[Figure]
rameters, we can still ﬁnd our solution can be stable across different parameters and discover some regularity on parameter tuning.
B P ROOF OF T HEOREMS B. P ROOF OF T HEOREM
Proof.
To prove Theorem 1, we ﬁrst provide several notations on graph G and node v i to facilitate the analysis.
(1) d i serves as the degree of node v i while the feature value x i is bounded by B , (2) ρ ( W ) denotes the largest singular value in learnable parameters W , (3) the ratio of intra-graph spatial homophily π sij is ﬁxed to p i , while the ratio of other node-level relations is q i , where q i = 1 − p i , (4) assume a target coefﬁcient γ i quantiﬁes the relationship between the targeted step T + 1 and current step t by x T
+1 i
= γ
i x ti .
We omit the non-linear activation layer for simpliﬁcation, as it will not intervene this theoretical analysis.
Based on above assumptions, we can introduce the following
Lemma 1 to demonstrate the representation h i will ﬁnally converge to its expectation.
Lemma 1 Acorrding to Hoeffding’s Inequality Bentkus (2004), the aggregated representation h i will converge to its expectation E [ h i ] with their differences bounded by,
[Equation]
Lemma 1 has been exactly proved in literature Ma et al.
(2021) on classiﬁcation tasks.
Considering
GNNs share the same process of neighborhood aggregation and node representation updates on both classiﬁcation and regression tasks, Lemma 1 can be extended to regression tasks.
After then, we can derive the expectation E [ h ti ] of h ti to complete following analysis.
For simpliﬁcation, we only consider the evolution from the one step T to the targeted step T +1 , i.e., x Ti → x T
+1 i , where we assume x T +1 i
= γ
i x Ti .
If only the nodes of intra-graph homophily can be aggregated, each aggregated neighbor v j ∈ N ( v i ) satisﬁes x j < (1 ± ε ) x i .
Let x j = (1 ± (cid:101) ε ) x i , where (cid:101) ε satisfy (cid:101) ε < ε < 1 .
Therefore, we can calculate the expectation E [ h Ti ] after once convolution, i.e., [Equation]
[Equation]
where d i p i interprets the expectation number of the spatial homophily nodes in the whole graph.
Given the target coefﬁcient γ i , a successful aggregation can be equivalent to ﬁnding a valid d i to construct an appropriate neighborhood ﬁeld for achieving targets.
Therefore, we can continue to derive the relationship between d i , γ i , and p i .
( p 𝑖 ,0) 𝜸 𝒊 𝒅 𝒊 𝑑 𝑖 =-
The difference between E [ h Ti ] and the targeted value x T [-+
-]+1 i
= γ
i x Ti can be written as,
[Equation]
Consider that the differences will converge to 0 as the optimized points are obtained, we can derive the closed solution of d i ,
[Equation]
By analyzing the above inverse proportional function in Figure 10, we achieve the conditional relationship among the substructure d
i , the target coefﬁcient γ i and intra-graph homophily p
i ,
[Equation]
Based on above analysis, we have theoretically demonstrated that we cannot obtain the optimized d i when γ i < p
i and γ i > 1 , corresponding to the discordance between topology and tasks.
(cid:3)
B. P ROOF OF T HEOREM
Proof.
To faciliate the analysis on more complex message passing, we introduce the following notations and settings.
Given node v i , the spatial homophily still keeps as p i while heterophily ratio keeps as q i , d i q
i is the expectation number of heterophily neighbors to v i , and α adjusts the aggregation proportions between homophily and heterophily.
We consider the expectation values of positive and negative heterophily neighbors respectively as x i λ 1 and λ 2 x
i where ( λ 1 , λ 2 ) satisfying 0 < λ 1 , λ 2 < 1 are the scalar factors.
By formulating a seperated aggregation strategy, we can derive the difference between the expectation of currentgraph representation and target as,
[Equation]
with
[Equation]
Let the difference be 0 , we have the following equation,
[Equation]
Ignoring the relatively small ε , we can arrive the optimzed closed form of d i ,
[Equation]
i [Equation] To ﬁnd an optimized d i ∈ N + , the parameter tuple ( p i , q i , α, γ i , λ 1 , λ 2 ) must conform to the following inequality by simplifying s p = 1 , s q = − 1 ,
[Equation]
Without loss of generality, we denote τ ∗ =
αλ 1 − (1 − α ) λ 2 and substitute q i with 1 − p
i , then we can simplify Equation 23 into,
[Equation]
Then we can derive the condition that τ ∗ must satisfy as,
[Equation]
Since the node-wise aggregation process are parameterized by both signed kernels and learnable parameters, it can be viewed as encapsulating learnable W into our pre-deﬁned parameters ( λ 1 , λ 2 , α ) and hence we can always ﬁnd the optimal τ ∗ to satisfy above inequality.
We will further present a detailed analysis of above inequality for verifying the existence of such solution.
Let τ ∗ =
t − 1 t , we have,
[Equation]
p
[Equation]
Then we can resolve the inequality and derive the exact t by,
[Equation]
Ultimately, a set of possible solution ( λ 1 , λ 2 , α ) to the directional message passing should satisfy the following inequality,
[Equation]
Given that λ 1 and λ 2 respectively imply the neighborhood of intra-graph positively heterophily and negative heterophily, in the following section, we will continue to analyze what conditions will λ satisfy when γ i tends to approach respectively 0 or + ∞ .
This will provide insights into the evaluations of predictability and GNN success on the given datasets.
1) when γ i → 0 , we have,
[Equation]
To ensure the a valid d i with 1 ≤ d i ≤ + ∞ , we derive,
[Equation]
We can obtain the constraints of λ 2 ,
[Equation]
Thus, for γ i → 0 , the successful aggregation can be achieved when compositions of negatively heterophily neighbors dominate the neighborhoods, i.e., we have a large λ 1 but small λ 2 satisfying
Eq. 31.
2) when γ i → + ∞ , we have,
[Equation]
To ensure the 1 ≤ d i ≤ + ∞ , we derive,
[Equation]
As the assumption of γ i → + ∞ but 0 < α, λ 2 , p i , q i < 1 , we can directly neglect the components of both p i and (1 − α )
λ 2 q i .
We then have,
[Equation]
Thus, for γ i → + ∞ , the successful aggregation can be achieved when compositions of positively heterophily neighbors dominate the neighborhoods, i.e., we have a small λ 1 but large λ 2 satisfying
Eq. 34.
Therefore, we can conclude that extending the aggregation to signed message passing with separately aggregating different types of neighborhoods can exactly enhance the aggregation capacity. (cid:3)
Remark. In Theorem 2, we demonstrate the required conditions of expected neighbors that potentially lead to successful aggregations under signed message passing.
With these derived conditions, we can roughly estimate whether the GNN can success or not to aggregate the neighborhoods for achieving targets if the dataset is given.
Based on Theorem 2, we can remedy the task-topology discordance by disentangling the neighborhood relationships and performing signed bi-directional message passing.
B. P ROOF OF T HEOREM
Intuitive demonstration.
As illustrated in Figure 11, we can observe that non-identical neighborhood environments can directly result in different optimized aggregation steps (orders) on each neighboring direction.
In other words, the neighborhood topology and the target-heterophily nodes determine the stopping step of aggregation along each route.
We demonstrate the detailed proof regarding their quantitative relationship as below.
Propagation
A ሚ𝐴
Propagation ෨𝐵
B 18 A
B
T
T+
Proof.
Due to the mixed compositions of intra-graph homophily and heterophily in one graph, there must be target-heterophily edges.
Intuitively, the targetoriented neighboring nodes, i.e., target-homophily are more beneﬁcial to the aggregation process leading to targets.
With the notations and basic setting in Theorem 3, we can analyze the expected number of total target-homophily neighbors.
To simplify the proof, we leverage local dependence assumption in Graph Information Bottleneck (GIB) theory Wu et al.
(2020a) in our analysis, which assumes nodes in a same k -order
EgoNet share the same neighborhood distributions and environments.
Also, herein we only consider a twolayer propagation ( k =2) and let the two-order neighborhood share the same local neighborhood environments ( d i , p, q p , q n ) speciﬁed by a central node v i (we will not impose the node index subscript to ( p, q p , q n ) as we make the local dependence assumption in the local ﬁelds).
We ﬁrst consider the scenairo of a large γ i in a consecutive transition x T +
i
[Equation]
>
[Equation] , and there are two cases that nodes are beneﬁcial to aggregate.
The beneﬁcial nodes are distributed hierarchically and carefully depicted in Figure 12(a), i.e., (1) in the ﬁrst-order propagation, the positively intra-graph heterophily neighbors to central node (circle colored in green), (2) in the secondorder propagation, the positively intra-graph heterophily compositions neighboring to ﬁrst-order homophily neighbors (left ellipse colored in blue), and the intra-graph homophily and positive heterophily compositions neighboring to ﬁrst-order neighbors that are positively heterophily compositions to v i (right ellipse colored in blue).
Let N + ( v i ) be the expected number of target-homophily neighbors of v i , we can derive the following equation, [Equation]
[Equation]
Denoting the expected feature values of ﬁrst-order target-oriented neighborhood as x o 1 and feature observations of the second-order neighborhood as x o 2 , the expected aggregation observations normalized by the degree d i can be derived as,
[Equation]
Therefore, we consider the information aggregated from target-oriented nodes as the probability that beneﬁcial nodes are exactly selected for aggregation.
In this way, the difference of such probability between the second layer and the ﬁrst layer ∆ P ( N + ) | 1 − 2 is computed by,
[Equation]
We consider the quadratic function of q p , where the function describes the compositions of targethomophily.
If 0 < q p < 1 1+2 η , we have ∆ P ( N + ) > 0 .
It refers to that when it is with a small targethomophily, the second-layer can gain more information than the ﬁrst-layer does,
i e., we have INFO ( h 2 i ) > INFO ( h 1 i ) .
Similar conclusion can be extended to INFO ( h 4 i ) > INFO ( h 3 i ) for 3 -rd and 4 -th layer propagation, and further higher-order propagations if they share similar neighborhood environments.
This numerical result can support the intuitive understanding that the smaller targethomophily is, the larger layer propagation steps are required to gain more informative target-related messages.
If q p > 1 1+2 η , we have ∆ P ( N + ) < 0 .
This interprets that a large target-homophily can encourage the ﬁrst-layer to gain more information than the second one, i.e., we have INFO ( h 1 i ) >
INFO ( h 2 i ) , and similarly INFO ( h 3 i ) > INFO ( h 4 i ) achieves.
An intuitive understanding can be achieved that the larger target-homophily, the fewer propagation steps are demanded to balance the propagation efﬁciency and information gains.
For the scenario of small γ , the negatively intra-graph heterophily compositions are encouraged to aggregate, as illustrated in Figure 12(b).
Now the proportion of target-oriented composition becomes q n in the ﬁrst-order propagation.
Let N − ( v i ) be the expected number of target-homophily neighbors of v i , we have,
[Equation]
Similar to above analysis, the probability difference of aggregating the nodes that are target-oriented (intra-graph negatively heterophily) between the second and ﬁrst layers can be derived as,
[Equation]
Eq. 39 is highly analogous to Eq.
37, hence the symmetric results can be achieved when we have a small γ i [-where x T +
-]satisfying γ i < 1 .
We can ﬁnally complete the proof of the quantitative relationship between the informativeness of layer-wise node representation and the target-oriented homophily ratio.
(cid:3)
B. B ROADER IMPACTS AND ETHICS ISSUE
Scalability and broader impacts.
In general, this paper proposes a novel GNN architecture, GReTo, for dynamic graph regressions via remedying the topology-task discordance.
Our work provides a new perspective that exploits both original topology and targets to investigate topology reﬁnement and reconstruction, i.e., the topology in structured data must be an informative knowledge for message passing while the aggregations may not involve all connected nodes but only the targetbeneﬁcial neighborhoods.
This research delves into how ﬁne-grained analysis of neighborhood compositions facilitates dynamic graph regressions and can promisingly inspire better modeling and analysis on various graph-structured data with complex explicit or implicit correlations. In detail, 𝑝 𝑞 𝑛 Hetero+ Hetero- Homo 𝑝 𝑞 𝑛 𝑝 𝑞 𝑝 𝑞 𝑛 𝑝 𝑞 𝑝 𝑞 𝑛 (a) 𝜸 is large 𝑞 𝑝 𝑁 + (𝑣 𝑖 ) =
𝑞 𝑝 𝑑 + 𝑝𝑞 𝑝 𝑑 2 + (𝑞 𝑝 𝑑 · (𝑝 + 𝑞 𝑝 )𝑑) 𝑝 𝑞 𝑛 Hetero+ Hetero- Homo 𝑝 𝑞 𝑝 𝑞 𝑛 𝑝 𝑞 𝑝 𝑞 𝑛 𝑝 𝑞 𝑝 𝑞 𝑛 (b) 𝜸 is small 𝑞 𝑝 𝑁 − (𝑣 𝑖 ) =
𝑞 𝑛 𝑑 + 𝑝𝑞 𝑛 𝑑 2 + (𝑞 𝑛 𝑑 · (𝑝 + 𝑞 𝑛 )𝑑)
[Figure]
it not only can be a good baseline for multivariate series prediction such as smart grid prediction, numerical weather forecasting, and economic growths, but it can also be generalized to resolve various classiﬁcation tasks on dynamic graphs including link predictions in social networks, citation networks, and recommendation systems.
Limitations and future works. The limitation of our work can be summarized as 1) limited direction expressivity for multi-step predictions, 2) lacking individual-level node analysis and interpretability on controllable message passing, i.e., dissecting how each composition of neighborhood affect the representation, deviating or approaching targets? Therefore, for future works, we are going to comprehensively study 1) how to exploit homophily theory to improve more general dynamic graph learning tasks such as link predictions or edge-type predictions, 2) how the node-level relations and neighborhood compositions affect the node representations, which potentially can control the message passing in a robust and anti-noise manner. Fairness and ethic issues.
Our work performs extensive analysis and experiments on datasets including trafﬁcs on two cities and climate datasets concerning air quality and temperature, without any personal identity and privacy issues.
Therefore, our work is with no ethics and privacy issues.
In addition, all baselines and methods are compared with fairness.