Heterophily-informed Message Passing
Authors: Haishan Wang
Arno Solin
Vikas K Garg
Venue: Accepted by TMLR ()

Abstract: Graph neural networks (GNNs) are known to be vulnerable to oversmoothing due to their implicit homophily assumption. We mitigate this problem with a novel scheme that regulates the aggregation of messages, modulating the type and extent of message passing locally thereby preserving both the low and high-frequency components of information. Our approach relies solely on learnt embeddings, obviating the need for auxiliary labels, thus extending the benefits of heterophily-aware embeddings to broader applications, e.g. generative modelling. Our experiments, conducted across various data sets and GNN architectures, demonstrate performance enhancements and reveal heterophily patterns across standard classification benchmarks. Furthermore, application to molecular generation showcases notable performance improvements on chemoinformatics benchmarks.

Published in Transactions on Machine Learning Research (04/2025)
Heterophily-informed Message Passing
Haishan Wang
haishan, haishan.wang wwngwoalto.
Aolto University
Arno Solin Aalto University
arno.solin@oalto..
Vikas Garg
ugary@csail. mit. edu
YaiYai Ltd and Aalto University
Reviewed on OpenReview: https: // openreview. net/ forun? id- 9fPinePPiii 4H2
Abstract
Graph neural networks (GNNs) are known to be vulnerable to oversmoothing due to their implicit homophily assumption. We mitigate this problem with a novel scheme that regulates the aggregation of messages, modulating the type and extent of message passing locally thereby preserving both the low and high-frequency components of information. Our approach relies solely on learnt embeddings, obviating the need for auxiliary labels, thus extending the benefits of heterophily-aware embeddings to broader applications, e.g. generative modelling. Our experiments, conducted across various data sets and GNN architectures, demonstrate performance enhancements and reveal heterophily patterns across standard classification benchmarks. Furthermore, application to molccular generation showcases notable performance improvements on chemoinformatics benchmarks.
1 Introduction
Methods for ubiquitous graph-structured data with abundant topological information have advanced the field of graph representation learning in recent years. Graph neural networks (GNNs) have emerged prominent deep learning models in this domain (Hamilton, 2020). A key feature of GNNs is the message-passingg (MP) scheme inspired by belief propagation-wwe which facilitates the processing of local topology while maintaining computa- tional efficiency (Dai et al., 2016). The MP schome enables local information exchange between nodes and their neighbours; implicitly assuming strong homophily, i.e., the tendency of nodes to connect with others that have similar labels or features. This assumption turns out to be reasonable in settings such as social data (McPher- son et al., 2001), regional planning (Gerber et al., 2013), and citation networks (Ciotti al., 2016). However, heterophilous graphs cxist in many scenarios, e.g., in fraud transaction networks (Pandit et al., 2007) and actor co-occurrence networks (Tang et al., 2009). They violate the homophily assumption, leading to sub-optimal per- formance (Zhu et al., 2020; 2021; Chien et al., 2021; Lim et al., 2021; Wang et al., 2023), owing to oversmoothing (Li et al., 2018) resulting from flattening of high-frequency information (Wu et al., 2023) by MP schemes.
A conceptual way to characterize homophily by examining the neighbours of each node a graph. For example, in a graph representing molccule fully homophilous molecule only has links between atoms of the same lype, while a helerophilous molecule has links between different types. However, in practice, node labels necessary for homophily calculation are often missing due to lack of information unavailable due to privacy concerns. Instead, heterophily typically stems from more intricate properties of the graph which need to be learned from data. For this problem, we introduce a general seterophily-informen MP scheme to carefully address and utilize the heterophily properties in graph data.
Many previous works analyze the impact of heterophily on GNN performance and design innovative structures to mitigate it (Zhu et al., 2020; Liu et al., 2021a; Yan et al., 2022; Ma al., 2021). Some studies offer deeper insights into how heterophily affects model expressiveness (Ma et al., 2021; Luan et al., 2022; Mao et al.,

---

Published in Transactions on Machine Learning Research (04/2025)
2023; Luan al., 2023). However, most of these works provide specialized model and only focus simple node classification tasks. In contrast, we aim to design a simple yet flexible structure that can be applied to any GNN. We verify its effectiveness not only on classification tasks but also on more challenging molecular generation tasks, which roquire models to learn the data distribution on graph embeddings.
Our contributions We introduce novel heterophily-inforan MP scheme, serving as an approach for general GNNs to utilize data heterophily, designed to learn graph structures and node features across varying degrees of homophily and heterophily. The scheme improves various GNN archilectures for node classification in different domains of data. Furthermore, we apply our approach to flow-based graph generation model (HetF lows). Our key contributions are summarized below:
We propose an architeture-hndeenddeni approach that encodes homophily/ hhephlopyyhhttohilyyyyy. patterns for general GNNs with flexible applications and highlights the necessity of data heterophily as the model prior.
Conceptual and technical:
Methodological: We design hederophily-informed message-passingg scheme focusing on specific node heterophily patterns and apply it to (5) various classic GNNs for node classification and (ti) an invertible flow-based model (HetFlows) for molecule generation.
Empirical: We demonstrate the benefits of our idea by benchmarking node classification accuracy and in molecule generation by evaluating the generated molecules with an extensive set of chemoinformatics metrics.
Notable advantages of our model include enhancing performance different graph learning tasks without adding extra parameters, offering flexible applications across various GNN architectures, and revealing the homophily pattern match between message passing and data sets.
2 Related Work
Graph heterophily Graph heterophily refers to the connectivity tendency among nodes with different labellings, as opposed to graph homophily. This property could be measured by node homophily (Pei et al., 2019), edge homophily (Zhu et al., 2020), or more dedicated designed metrics (Lim et al., 2021; Platonov et al., 2023). Recent research shows the importance and benefits of considering heterophily in graph learning. For example, Empirical experiments (Zhu et al., 2022) show heterophilic nature in structure attacks, and validate that heterophily incorporation enhances model robustness to ad adversarial attacks. Similarly to oversmoothing, the heterophily challenges graph learning by less discriminative node representation, Bodnar et al. (2022) links the GNN performances in heterophilic graphs with the oversmoothing problem by cellular sheal theory.
Heterophily-awarred GNNs Numerous techniques have been developed address degradation in the performance of GNNs in heterophilic settings. Some approaches expand the concept of neighbour sets, such aggregating messages from farther hops of neighbours (Abu-El-Haiee El- et al., 2019; Wang & Derr, 2021) or scarching for potential new neighbours (Pei et al., 2019; Jin et al., 2021; Li et al., 2022). Other focus on refining the message during the aggregation process, such as differentiating neighbours through specific filters (Luan et al., 2022; Yan et al., 2022; Wang et al., 2023) or collecting embeddings from previous layers, like jumping knowledge (Xu et al., 2018) and generalized PageRank techniques (Chien et al., 2021). These methods typically require specialized structures or ignore the local homophily during message passing achieve their effects. However, we hope to capture the local homophily difference with minimal structure modifications, making the solution flexible and widely applicable. Heterophily- Meteroeeily-informeed MP can be viewed performing adaptive message modulation before aggregation.
Molecule representation and generation Early works in molecule generation (Kusner et al., 2017; Guimaraes et al., 2017; Gómez-Bombarell et al., 2018; Dai et al., 2018) primarily used sequence models encode the SMILES (short for 'Simplified Molecular-Ini Input Line-Entry System') strings (Weininger al., 1989), posing generation autoregressive problem. However, the mapping from molecules to SMILES is
2

---

Published in Transactions on Machine Learning Research (04/2025)
not continuous, similar molecules can be assigned vastly different string representations. Graphs provide elegant abstraction to encode the interactions between the atoms in a molecule. Thus the field has gravitated towards representing molecules as (geometric) graphs and using powerful graph encoders; e.g., based on graph neural networks (GNNs), for example, adversarial models (De Cao & Kipf, 2018; You et al., 2018), energy-based models (Liu et al., 2021b), diffusion models (Hoogeboom et al., 2022; Xu et al., 2023), Neural ODEs (Verma et al., 2022) and flow-based models (Shi al., 2019; Luo al., 2021; Zang & Wang, 2020).
3
Heterophily-informee Message Passing
We propose an archilectue-inddeendeel approach that encodes snnoppilyyhhterophiyy patterns for general GNNs and later apply it to GNNs in node classification and graph generation setups.
3.1 Prerequisites: Message Passing
Graph Neural Networks (GNNs) have emerged as potent paradigm for learning from graph-structured data, where the challenges include diverse graph sizes and varying structures (Kipf & Welling, 2017; Veličković et al., 2018; Xu et al., 2019; Garg et al., 2020). Consider a graph G = (V, E) with nodes V and edges E. For these nodes and edges, we denote the corresponding node features as X = { E IR= | u V} and edge features as E fevv E R u, u E E}. Here ne, ny are the feature dimensions of nodes and edges. For each node e V, its embedding at the 8th layer is represented as (R) and !a)) = (h. (R) V V}. These embeddings evolve through a sequence of transformations across GNNs of depth K, by the message passing scheme (Hamilton, 2020):
m MESSAGE( (N.7) eun) WE h (k UPDATE (h (14))m))1))) m
(1)
(2)
for & E {0, 1,..., K3. Here ((v) denotes the neighbour set of node v. Both UPDATE(3) and MESSAGE()) are arbitrary differentiable functions. Messages from all neighbours of e are aggregated in the multiset (k)
m ((t) = (mm (4) u J (uu)). Importantly, the function UPDATE needs to be permutation invariant on
this message set m (k)
Wt)) (e.g., by resorting to operations like summation taking the maximum). However, in practice, naive aggregation strategy typically mixes messages, especially in heterophilie locality, leading to the 'oversmoothing' problem (Zhu et al,, 2020; 2021; Chien et al., 2021; Lim et al., 2021; Wang et al., 2023).
3.2 Heterophily-informmet Message Passing
Our method encodes the heterophily assumption into the MP scheme of the GNN, denoted as GNN (X E) = (K), The indicator r = forig., hom., het.] specifics the heterophily preference of the GNNs as a hyperparameter: whether they lean lowards homophily (hom.), heterophily (het.), and original structure (orig.). The GNNorig. exactly the original GNN described at Sec. 3.1. And we name GNNhet. and GNN after the HctMP and HomMP modes of GNNorig, Referring to Eq. (1) and Eq. (2), the messages undergo different scaling preprocessing steps before being sent forward to the subsequent layer:
(k) MESSAGE()) (....e.) m kr
UE uE N(e),
(3)
m ((1))
(N)
Ho m (R) |uEN(
(4)
h (R+1)
(PDATE(() (N(.) (10)m//i -mm))))))
(5)
where the scaling factors
H(u,v),
if 7 hom. if 7 = orig.
(1)
1,
(6)
H(u, v),
7 het.
3

---

Published in Transactions on Machine Learning Research (04/2025)
3-Aminophenol
Model input
Heterophilious channel
Original channel
Homophilious channel
Leya
Figure 1: Comparison of hederophill--informed MP with original GNN on graph describing the 3- Aminophenol molecule. The three channels show how controls the scaling factor in Eq. (6) and leads to different message passing behaviour, given the same input.
and H denotes the node homophily and the embeddings are initilized by h( = X. Aiming to learn embeddings as node labels, therefore in practice, instead of traditional label-style definition many contexts, we define the homophily or attraction to similarity between embeddings as the cosine similarity H(u, v) & Scos (h (k) at the relevant layer in Eq. (6). For more representative embeddings, three channels (HomMP, P, HetMP, and original model) could be combined with an additional linear layer be a mixed model (mix.) named MixMP. App. A.5 provides an example of GCN-based MixMP.
The message passing process of example GNN and its HetMP, HomMP versions are visualized in Fig. 1 three channels. The input is an example molecule 3- 3-Aminophenol, and node colouring corresponds to the embeddings while similar colour means closer embeddings. The thickness of the bond corresponds to the scaling factor at, (which shows the similarity between node pairs) of the last layer. The homophilous channels (hom.) decrease information exchange between dissimilar neighbours, introducing frictions during message flows. The hetcrophilous channel (het.) encourages faster message spreading on higher frequency locality, while slowing it down when neighbors become similar. In this simple example, the original channel (orig.) shows the typical oversmoothing issue: all the node embeddings become similar after layers, while the other two channels (homophilous/heere haaoopppllossheterrphhhoooo heterophilouu channels) mitigate it with different convergence resistances, thus improving model effectiveness.
3.3 Application to Molecular Generation
For generative modelling on molecule graphs, we propose a graph-generation model based on heterophily- informed MP: HetFlows. The model built on a normalizing blow- model MoFlow (Zang & Wang, 2020), which generates molocules with estimated probability and ensures uncertainty estimation and applications of Bayesian tools. Our model is split into two main components: a bond flow and an atom flow. The bond flow learns the molecular topology, while the atom flow assigns certain atomic details to this structure.
Prerequisites In general, normalizing flows offers methodological approach to model distributions based on the change-of-variable law of probabilities. This is achieved by applying a chain of reversible and bijective transformations between trivial variables (like Gaussians) with target data variables and updating the transformation to minimize the negative log-likelihood (Dinh et al., 2014).
Affine coupling layers (ACLs) introduce reversible transformations to normalizing flows, ensuring efficient computation of the log-determinant of the Jacobian (Kingma & Dhariwal, 2018). ALCS keep reversibility
4

---

Published in Transactions on Machine Learning Research (04/2025)
by updating partial information via the other part of the information. Scries of ACLs (fi...., fr} build reversible flow between the Gaussian with the target distribution / fr .... f1.
Training and loss Given molecule graph G = (X, E), the atom flow fa and bond flow fa map the graph into embeddings which follow Gaussian distributions Nb:
fa(X E)= ha ~aa f((E) = hs NN-
The model is trained to minimize the negalive log- liglliies (NLLs) of the data by gradient descent. The target loss function of the model L could be decomposed into two parts since HetFlows contains two flows L = Lu + L.- The atom loss Lu is defined following
fa ox
C. loggx (X) log (ha) log det
(8)
The bond loss L. is defined similarly as above.
Generation process Given a trained model Jas:fex with established parameters, sampled embeddings randomly from Gaussian h hy -N,, Then the embeddings can generate features of bonds E = fia (hs) and atoms X fae (h, E) in sequence, which requires the reversibility of flow model. Finally, the molecules are reconstructed G = (X, E). Additionally, the bond features generation can be achieved by sampling the adjacency matrix (denoted by '+true adj.' later) from the real data distribution which reflects the model separability.
The HetFlows shares the same structure of MoFlow (Zang & Wang. 2020): ACT-based normalizing flows. The coupling function stores most of the paramcters of an ACL, which serves as the most essential part. The main difference between these two methods the message passing scheme of GNN utilized as coupling functions in all ACLs of flow: MoFlow contains the classical graph convolutional networks (Kipf & Welling, 2017), and the HetFlows substitute it with a MixMP version of it. Further details of HetFlows are discussed in App. A, including introductions of normalizing flows and ACLs, mathematical model description, the training and generation process of Het Flows, the loss function, and proof of model reversibility.
4 Experiments
We demonstrate the effects of hetcrophily-infonee MP both in discriminative node classification benchmarks and molecule generation settings. Sec. 4.1 compares the node classification performances of various types of GNNs with their variants (HetMP, HomMP, and MixMP versions) across 15 data sets. In Sec. 4.2, we demonstrate how our MixMP GCN blocks improve flow-based molecule generation, directly showing the benefits of our MP scheme.
4.1 Node Classification Benchmarks
Data sets The node classification experiments belong to the class of semi-supervised learning tasks. We evaluated on homophilic data sets in citation networks (Yang al., 2016) (CORA, PUBMED, CITESEER) and co-purchase graphs (Shehur et al., 2018) (COMPUTERS, PhOTo). Furthermore, the 10 heterophilic data sets including hyperlink networks (Pei et al., 2019) (CORNELL, WISCONSIN, TEXAS), Wikipedia networks (Rozemberezki et al., 2021) (CHAMELEON, SQUIRREL), and heterophilous graph dataset (Platonov et al., 2023) (ROMAN-EMPIRH A AMAZON-RATINGS, MINESWERPER, TOLOKERS, QUESTIONS). Here the graph homophily Hes is measured by class insensitive edge homophily ratio (Lim et al., 2021).
Setups The models were implemented PyTorch (Paszke et al., 2019) and PyTorch Geometric (PyG) (Fey & Lensson, 2019). The basc model GNNs in Sec. 4.1 are the PyG-implemented versions. Each one and variants (HetMP, HomMP) contain 2 layers and 128 dimensions for all hidden layers. All the models are trained with the AdamW optimizer (Toshchilov & Hutter, 2019), learning rate 0.001 and drop-out ratio 0.2.
5

---

Published in Transactions on Machine Learning Research (04/2025)
Table 1: We shed light on how our homo-/heterophilove message passing can boost standard node classification benchmark accuracy (std) Data sets are sorted according to their homophily level Hes For each data (column), four classic GNN architectures (GCN, GAT, GIN, GraphSAGE) are tested, grouped with their corresponding HctMP /HAMMPP/HMMM//// /MixMP modes (with +hom., thom. +mix.). Numbers are bolded with paired t-test (5% significance level) for each 4-mode group. Our MixMP models perform either equally well or significantly better in all cases, with an average of 3.84 (7-points) accuracy improvement than the original model. Note that the accuracy of binary class datasets is reported as their ROC-AUC scores, which follows the convention from Platonov et al. (2023).
Henoppilty
0,000
0.31
0.03
0.00
0.005
0.05
0.38
0.51
0.65
1.70
0.77
GON
58:4
12,5
47.8
26:31L:1
42.7
37.3134
68.2
52.4114
49.1.00
58.0112
T0.7
85.5183
91.0003
15.0
14.4 0J
CCN louz.
55 6 5.
07.5
16.0 06
20.0 23.611
61.4 43.4155
40:4135
$2.2193
04.5123
47-1-103
87. 07.7:35
T0.511-4
88.7103
01.0
94
GCNN-mit
56 8m
54.5
92.0g17
306 9151
59. Tx
7-
T6.9
88.6g0
NT.7aI7
CAT
57
47-1-06
29 411.5
43.7
45 45.3154
12.4184
05.8003
07.005.
10.3.11
27.1
01.3 01.2000
50.0.00
CAT hom.
59 5.
66. 6.5.
07-0.12
82 22:54L5
47.01125
48:5134
55.0193
$4.915
47.2.07
60. 7.30
16-9.1-4
88 88:2103
30:3183
$5.0.LM
01.0..0
68.0.13
01..1
27:3114
51.3184
63.3144
00.
07.01117
70.0531
24.0114
$7.8.-1.
GrapheSACat
76 9:5.4
365.5
50.:2
56 56.0
18: 1203
10. 1
NT
CraplSACI lotu.
70.6
04:42144
78 8116
30.211 2214
07 8:3-1
60.00 8124
36.7
60 2012
03.0333
50
17 4:007
88 88.0
30 5
67 571.1
$264
CeaplSACCB let.
T1.0.19
18.1
9:16
50.722
56.
87. 73.5
06.0113
21
2114
90
GLN
06.2-13
26. 3633
36..
$1.8
44.0-1.
50.2
53.3
T1.0:13
06: 1814
41 2631
$4.0-111
50.3-14
GIS hem.
56 1-43
24.2-44
65.9:12
28 03.0
48.55 8-
360 26.0021
06. 7-14
40.0113
40.2-A7
53 0
T1 T1.2
NT.4
42:8111
84.3-13
57.T
GIN -et.
59.5 5-30
7:83
56.6-11
50.3--11 26.5.22
35.9
30.8124
02:0113
48-2-1.3
53-7:14
00:3311.7
81
43:1-44
81.9
GCN
het-arig
GAT
ahet - Dorig & apon - Dotig
orig
GraphSAGE
GIN
0.2
0.4
0.9
0.8
0.2
0.4
0.6
0.8
Data set homophily Hei
Data set homophily Hei
(a) Accuracy improvement of MixMP
(b) Accuracy improvement of HetMP and HomMP
Figure 2: Comparison over 15 data sets and 4 GNN architectures in terms of improvement in accuracy (%-points). Gorig, (homs Ohet and Omix denote the accuracy of original model and corresponding HomMP, HetMP, and MixMP versions. In Fig. 2a, each node the average performance over 10 random seeds for a specific dataset and GNN structure, illustrates the accuracy advantage of MixMP above the original model. In Fig. 2b, the y-axis denotes the accuracy improvement of HetMP and HomMP over the original structure.
The data split settings is tested for 10 random model initializations and data splits.
= 60%/20%/200) Each configuration (data set and model)
Baselines Four different GNN architectures are chosen as baseline models: Graph Convolutional Network (GCN) (Kipf& & Welling, 2017), Graph Attention Network (GAT) (Veličković et al., 2018), Graph Isomorphism Network (GIN) (Xu et al., 2019), and GraphSAGE (Hamilton et al., 2017). For more details about the data split rules and base GNNs, please refer to App. B.
The comprehensive benchmark results in Table include the node classification accuracy (or ROC- AUC score for binary labels), cach value representing the average of 10 random runs of the four base models and their HetMP, HomMP, and MixMP versions across 15 different graph domains, with data homophily displayed. For each GNN and data set, the best result among the four modes is highlighted in bold. For more intuitive visualizations, Fig. 2 illustrates the benefits of classification accuracy on heterophily-nffomeed MP and special patterns through the accuracy discrepancies. Further comparison with other heterophily-aware baseline models is in App. B.4.
Results
6

---

Published in Transactions on Machine Learning Research (04/2025)
Analysis Table demonstrates that, across 10 heterophilic data sets, our proposed MixMP performs comparably or better than the baseline in 38 out of 40 cases, evaluated over four types of GNN architectures. This strong performance is further supported by Fig. 2a, where the majority of data points lie on or above the y-axis, indicating that the MixMP consistently enhances node classification performance. Notably, the improvement is stronger on heterophilous datasets compared to homophilous ones (with minor outliers from GIN), suggesting that our model modification is able to incorporate data heterophily as structural prior without benefits from homophilic data. The consistent performance gains across diverse setting combinations underscore the potential generalization capability of our approach, which could also be applied to wider range of GNN structures and data domains.
In Fig. 2b, the accuracy differences Obet - Cori and ahom Gori are mostly concentrated in the top-left and bottom-right regions. For homophilic data, the HomMP exhibits a minor impact on accuracy, whereas the HetMP even adversely affects performance. In contrast, heterophilic data. benefit from both the HomMP and HetMP on the node classification task. These observations provide empirical evidence for the implicit homophily assumption underlying the traditional MP scheme. Furthermore, both the HomMP and HetMP mitigate the issue of oversmoothing in heterophilic data, as visualized in Fig. 1.
Minor performance improvements are observed on the CORNELL, WISCONSIN, and TEXAS datasets, likely attributed to their small graph size (183-251 251 nodes) with limited statistical significance. In contrast, our MixMP achieves substantial significant improvements on the MINESWEEPER and ROMAN-EMPIR which are specifically designed to evaluate GNNs under heterophily (Platonov et al., 2023). These results further validate the advantages of our approach in scenarios where heterophily is a critical factor.
In conclusion, the node classification experiments highlight the capability of MixMP to incorporate data heterophily as prior and euhance node-level graph represenlation learning, especially on heterophilic data.
4.2 Molecule Generation
Molecules express varying levels of homo-/heterophily. Thus molecular modelling provides an interesting benchmark for our proposed MP scheme. We demonstrate the impact accounting for heterophilic message passing in a variety of common benchmark tasks for molecule generation and modelling. We provide results for molecule generation with benchmarks
wide range of chemoinformaties metrics.
Setups The HetFlows in Sec. 4.2 is built on GNNs with 4 layers and flows that were li = 27, = 10 (for QM9) deep and ka 38, h 10 (for 2INC-250K). For the generation task, we select the best-performing model using the FCD score as suggested in Polykovskiy et al. (2020). We report numbers without post hoc validity corrections.
Data sets We consider two common molecule data sets: QM9 and ZINC-250K. The QM9 data set (Ramakrishuan et al., 2014) comprises ~ 134k stable small organic molecules composed of atoms from the set {C, H, O, N, F}. These molecules have been processed into their kekulized forms with hydrogens removed using the RDkit software (Landrum et al., 2013). The 2INC-250K (Irwin et al., 2012) data contains ~250 drug-like molecules, each with up to 38 atomns of 9 different types.
Chemoinformaties metrics We compare methods through an extensive set of chemoinformatics metrics that perform both sanity checks (validity, uniqueness, and novelty) on the generated molecule corpus and quantify properties of the molecules: neighbour (SNN), fragment (Frag), and scaffold (Scaf) similarity, internal diversity (IntDiv, and IntDiv2), and Fréchet ChemNet distance (FCD). We also show score histograms and distribution distances for solubility (logP), synthetic accessibility (SA), dtrug-likneee (QED), and molecular weight. For computing the metrics, we use the MOSES benchmarking plat.frrm (Polykovskiy et al., 2020) and the RDKit open-source cheminformatics software (Landrum et al., 2013). The 'data' row in metrics is based on averages over randomly sampled sets (1000 mols per set) from the data. For the metrics, we simulate 10 batches of 1000 mols and compare them to hold-out reference set (20% of data, other 80% used for training). Full details on the 14 metrics we use are included in App. D.
7

---

Published in Transactions on Machine Learning Research (04/2025)
Frstt weetto--too toe 4Q,
Mortin Porton 1or
125
200
logP
SA
QED
Molocular weight
Figure 3: Chemoinformatics statistics for data (QM9) and generated molecules from HetFlows (ours), MoFlow, and GraphDF. We report histograms for the Octanol-water partition coefficient (logP), synthetic accessibility score (SA), quantitative estimation of drug-likeness (QED), and molecular weight.
Table 2: Chemoinformaties summary statistics for random generation the QM9 molecule data. Full listing of all 14 metrics in Table A6. The results show that our message passing modification of MoFlow (resulting in HetFlows) achieves better results on FCD, SNN, Frag, and Scaf, and retains competitive performance on other metrics.
FCD
Validity 1
Novelty
SNN
Frag
Scaf
IntDiv f
Data (QM9)
0.40
1.00.0.00
0.6220.02
0.5420.00
0.94.000
0.7620.08
0.9220.00
GraphDF
10.76
0.98-0.00
0.35=0.00
0.61=0.01
0.09
0.87:10.00
MoFlow
7.55
0.95
0.9620.0.
0.32=0.00
0.6020.05
0.0420.03
0.92:0.00
FotFlow (Ours)
4.04
0.92:0.0:
0.92:0.0:
0.34=0.00
0.80=0.02
0.04
0.91:10.00
MoFlow true adj.
4.45
1. 2.0000000
0. 0.85
0.38_0.00
0.70
0.31
0.92:0.00
HetFlow true adj.
1.46
1.00
0.74:0.:
0.43
0.85
0.52
0.92:0.00
Table 3: Chemoinformatics summary statistics for random generation on the ZINC-250r molecule data set. Full listing of all 14 metrics in Table A7.
FCD 4
Validity
Novelty f
SNN 1
Frag
Scaf
IntDiv,
Data (ZIvo-250k)
1.44
1.0020.00
0.0220.00
0.5120.00
1.00g0.00
0.28_0.02
0.87
Ploes
GraphDF
34.30
1.00+0.00
0. 0.03
0.35-
0.00-0.0
0.88+
MoFlow
23.33
0.89=0.01
1.00:0.00
0.27=0.00
0.79
0.01-0000
0.88
HetFlow (Ours)
23.72
0.87-0.01
1.00
0.26-0.00
0.77-0.0
0.01-0.0
0.88 0.00
AbL.
MoFlow adj.
8.21
0.94
1.00
0.33:0000
0.80
0.07=0.02
0.87:10.00
FetFlow +ow-e ad].
8.24
0.93-0.01
1.00+0.00
0.34 +0.00
0.91
0.10
0.87++0.00
Basclines For random generation, include bascline results for methods that have pro-trained models
publicly available. Trained models are required for generating chemoinformatics metrics beyond trivial sanity checks (validity, uniqueness, and novelty). We compare GraphDF (Luo et al., 2021) and MoFlow (Zang & Wang, 2020) which are current statc-of-tho-art flow models for molccular gencration.
Results on QM9 For the QM9 data set, the main chemoinformatic summary statistics are given in Table and the descriptive distributions in Fig. HctFlows +true adj. achieves best validity, SNN Frag and Scaf, especially lowest FCD over flow-based models. Full benchmark results are available in the extended listings in Table A6 in the Appendix.
Results ZINC-250K For the ZINC-250K data set, the main chemoinformatic summary statistics are given in Table 3 and the descriptive distributions in Fig. A7. While the ZINC-250k data set is more complicated, HetF'lows +true adj. achieves the best Novelty SNN, Frag and Scaf, and has competitive performance on other metrics. Full benchmark results are available in the extended listings in Table A7.
Analysis HetFlows emerges as a robust and versatile molccular gencration model, adept at balancing fidelity, diversity, and molecular properties. Notably, the validity on both QM9 and ZINC-250K are higher than 85%, ensuring the model's reliability. As discussed in Sec. 3.3, Floss is built. based on the MoFlow (Zang &
8

---

Published in Transactions on Machine Learning Research (04/2025)
6.5
0,33
0.00
0.30 0:38
0.00 0056 8.55 0:45
0:41 0.33 0.39 6:2 0.38
Figure 4: Structured latent-space exploration (QM9). Our approach yiclds a sensibly structured latent space as qualitatively demonstrated by this nearest neighbour search in the latent space with the seed molecule the left and neighbours with the Tanimoto similarity (1 0) given for each molecule. For results ZINC-250K, see Fig. A6 in the appendix.
Wang, 2020). replaces the classic graph convolution of the GCNs module of MoFlow with MixMP version. In other words, the difference between MoFlow and HetFllow is the GNNs, more exactly, the MP scheme. As Table 2 and Table 3 show, HetFlows is superior over MoFlow with both adjacency matrix generation styles QM9 and ZINC-250K. Fig. 3 shows the chemoinfornaties statistics of generated molecules from HetFlows fit the original distribution better than MoFlow in both generation styles. It provides evidence of the advantage of utilizing the heterophily inside the message passing, as intuiled by Fig. 1.
Visualizing the continuous latent space Inspired by Zang & Wang (2020), we examine the learned latent space of our method on both QM9 and ZINC-250K, presented in Fig. 4 and Fig. A6, respectively. Qualitatively, we note that the latent space appears smooth and the molecules near the seed molecule resemble the input and have high Tanimoto similarity (Rogers Hahm, 2010).
4.3 Ablations Studies
To further verify that the effects we see are due to our proposed heterophill--nffrmeed MP scheme, we include ablation studies on different adjacency matrix generation strategies and parameter sharing of the GNNs.
Ablation study: adjacency matrix generation As mentioned Sec. 3.3, the adjacency matrix can be generated by the bond model or sampled directly from the real distribution. Both approaches are present in literature as basis for modelling (see, e.g., discussion in Verma al., 2022). With sampled adjacency matri- ces, the main task of the model is to put correct labels given graph topology. As the comparison results in Table 2 and Table 3 show, the adjacency matrix generated by the bond model limits the model generation perfor- mance compared with the sampling alternative. However, this approach can be considered method of its own.
Ablation study: parameter sharing of MixMP GNN There are three channels of message passing scheme as shown in Fig. 1, the three channels could share parameters or not in the MixMP, which corresponds to heether MESSAGE(S in Eq. (3) is the same function for all E T or not. To investigate the improved expressiveness of MixMP GNNs from extra two channels, models with two settings are compared in Table A8 and Table A9. Based on the chemoinformatics metrics, the random generation outputs of the two settings are similar. It means the model strength stems from the more expressive structure intuited by Fig. 1, not from the larger parameter size.
9

---

Published in Transactions on Machine Learning Research (04/2025)
5 Conclusions and Discussion
We have prosented heterophily-informed meessag-ppssiingg scheme, flexible plug-in module of GNNs to account for heterophily prior, countering the traditional oversmoothing vulnerability prevalent in exisling GNN-based methodologies. By adjusting message passing to discern (dis-) (dis-)ssmilariiies between nodes, our method offers a more muanced representation of the intricate balance between affinitics and repulsions. In the experiments, we demonstrated our approach both in standard discriminative node classification benchmarks and by applying the approach inside a generative flow model (which we call HetFlows). Experiment results show the versatility and ability of the proposed scheme to enhance embedding expressiveness across multiple graph domains. The analysis underscores the necessity of aligning data homophily with corresponding model assumptions. We consider this approach promising tool in heterophilic graph learning.
One limitation of heterophill--uffrmmed MP is the variable effectiveness across datasets of different homophily levels. As discussed in Sec. 4.1, the scheme brings limited advantages to all homophily data and heterophily data of small sizes. It also suggests clear potential for improving various MP mechanisms. Such improvements could be achieved through better estimation of graph homophily and more intelligent integration of different. channels.
In molecular generation, the dependency on the current adjacency matrix generation (bond model) method perhaps restricts HetFlows's effects, as the key coupling functions, based on CNNs, capture limited structural information. With a sampled adjacency matrix, it successfully generales molecules that are valid, novel, diverse, and chemoinformaticall similar to those in the existing distribution.
A reference implementation of the methods is available at httpp://gttaab..ccmaaaaatt////pp
Acknowledgments
This work was supported by the Research Council of Finland (grants 342077, 362408, 339730), Saab-WASP (grant 411025), and the Jane and Aatos Erkko Foundation (grant 7001703). We acknowledge the computational resources provided by the Aalto Science-IT project and CSC - IT Center for Science, Finland.
References
Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. MixHop: Higher-order graph convolutional architectures via sparsified neighborhood mixing, In Proceedings of the 36th International Conference on Mochine Learning (ICML), volume 97 of Proceedings of Machine Learning Reseorch, pp. 21-29. PMLR, 2019.
Cristian Bodnar, Francesco Di Giovanni, Benjamin Chamberlain, Pietro Lio, and Michael Bronstein. Neural sheaf diffusion: A topological perspective on heterophily and oversmoothing in GNNs. Advances in Neural Information Processing Systems (NeurIPS), 35:18527-18541. 2022.
Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. Adaptive universal generalized pagerank graph neural network. In International Conference on Learning Representations (ICLR), 2021.
Valerio Ciotti, Moreno Bonaventura, Vincenzo Nicosia, Pietro Panzarasa, and Vito Latora. Homophily and missing links in citation networks. EPJ Data Science, 5:1-14, 2016.
Hanjun Dai, Bo Dai, and Le Song. Discriminative embeddings of latent variable models for structured data. In Proceedings of The 33rd International Conference on Machine Learning (ICML), volume 48 of Proceedings of Machine Learning Research, pp. 2702-2711. PMI.R, 2016.
Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax-directed variational autoencoder for structured data. Xix preprint 10x.:1802208786, 2018.
Nicola De Cao and Thomas Kipf. MolGAN: An implicit generative model for small molecular graphs. JCML 2018 Workshop Theoretical Foundations and Applications of Deep Generative Models, 2018.
10

---

Published in Transactions on Machine Learning Research (04/2025)
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation. arxiv preprint or Xix: 14 10.8516, 2014.
Matthias Fey and Jan E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
Vikas Garg, Stefanie Jegelka, and Tommi Jaakkola. Generalization and representational limits of graph neural networks. In International Conference Machine Learning (ICML), pp. 3419 3430. PMLR, 2020.
Elisabeth R Gerber, Adam Douglas Henry, and Mark Lubell. Political homophily and collaboration in regional planning networks. American Journal of Political Science, 57(3))598-66) 610, 2013.
Rafael Bombarl-ombarelli, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato Benjamín Timothy D Hirzel, Ryan P Adams, and Alán Aspuru-Guzik. Automatic chemical design using a data-driven continuous represcntation of molecules. ACS Central Science, 4(2):268-277) 276, 2018.
Jänchez-Lengeliing Dennis Sheberla, Jorge
Gabriel Lima Guimaraes, Benjamin Sanchez- Lengleiz,,nggiing, Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzit Objeeciiv--einforreed generative adversarial networks (organ) for sequence generation models. arXiv preprint Xrv: 17o::1705.0000 10843, 2017.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In A Advances in Nearal Information Processing Systems 30 (NeurIPS), pp. 1025- 1035. Curran Associates, Inc., 2017.
William I. Hamilton. Graph Representation Learning. Morgan & Claypool Publishers, 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling, Equivariant diffusion for molecule generation in 3d. In International Conference on Machine Learning (ICML), pp. 8867 8887. PMLR, 2022.
John J Irwin, Teague Sterling, Michael M Mysinger, Erin Bolstad, and Ryan G Coleman. ZINC: free tool discover chemistry for biology. Journal of Chemicol Information and Modeling, 52(7):175771768, 2012.
Di Jin, Zhizhi Yu, Cuiying Huo, Rui Wang, Xiao Wang, Dongxiao He, and Jiawei Han. Universal graph convolutional networks. In Advances in Neural Information Processing Systems 34 (NeurPPS), pp. 10654- 10664. Curran Associates, Inc., 2021.
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation. In International Conference on Machine Learning (ICML), pp.
PMLR, 2018.
Durk P. Kingma and Prafulla Dharival. Glow: Generative flow with invertible 1xl convolutions. In Advances in Neural Information Processing Systems (NeurIPS), volume 31, pp. 19236-10245 Curran Associates, Inc., 2018.
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR), 2017.
Matt J Kusner, Brooks Paige, and José Miguel Hernández-Lobalo. Grammar variational autoencoder. In International Conference on Machine Learning (ICML), pp. 1945 1954.. PMLR, 2017.
Greg Landrum et al. RDKit: A software suite for cheminformatics, computational chemistry, and predictive modcling. Greg Landrum, 8:31, 2013.
AA Leman and Boris Weisfeiler. A reduction of graph to a canonical form and an algebra arising during this reduction.
Teintnccsaenniihhsskaka Informatsiya, 2(9):12-16, 1968.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi- supervised learning. In Proceedings of the A AAT Conference on Artificial Intelligence, volume 32, 2018.
11

---

Published in Transactions on Machine Learning Research (04/2025)
Xiang Li, Renyu Zhu, Yao Cheng, Caihua Shan, Siqiang Luo, Dongsheng Li, and Weining Qian. Finding global homophily in graph neural networks when meeting heterophily. In Proceedings of the 39th International Conference on Machine Learning (ICML), volume 162 of Proceedings of Machine Learning Research, pp- 19242-13256, PMLR, 2022.
Derek Lim, Felix Hohne, Xiuyu Li, Sijia Linda Huang, Vaishnavi Gupta, Omkar Bhalerao, and Ser Nam Lim. Large scale learning non non-homophilous graphs: New benchmarks and strong simple methods. Advances in Neural Information Processing Systems (NewrTPS), 14:20887-20900 2021.
Meng Liu, Zhengyang Wang, and Shuiwang Ji. Non-local graph neural networks. IEFE Transactions on Pottern Analysis and Machine Intelligence, 44(12):10270 10276, 2021a.
Meng Liu, Keqiang Yan, Bora Oztekin, and Shuiwang Ji. GraphEBM: Molecular graph generation with energy-based models. arxiv preprint arXi::2102.005/4 2021b.
llya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019.
Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao Meo- Chang, and Doina Precup. Revisiting heterophily for graph neural networks. Advances in Neuval Information Processing Systems (NewrIPS), 35:1362 1375, 2022.
Sitao Luan, Chenqing Hua, Minkai Xu, Qincheng Lu, Jiaqi Zhu, Xiao Chang, Jie Fu, Jure Leskovec, and Doina Precup. When do graph neural networks help with node classification: Investigating the homophily principle on node distinguishabilityy arXiv preprint 21f::2304.14277, 2023.
Youzhi Luo, Keqiang Yan, and Shuiwang Ji. GraphDF: A discrete flow model for molecular graph generation. In International Conference on Machine Learning (ICMI.), pp. 7192-7203. PMTR, 2021.
Yao Ma, Xiaorui Liu, Neil Shah, and Jiliang Tang. Is homophily a necessity for graph neural networks?
XrXiv preprint arxi:2206.000 06194, 2021.
Haitao Mao, Zhikai Chen, Wei Jin, Haoyu Han, Yao Ma, Tong Zhao, Neil Shah, and Jiliang Tang. Demystifying structural disparity in graph neural networks: Can one size fit all? arXiv preprint Xrxvv:2306. 2tl.2330000123,, 2023.
Miller McPherson, Lynn Smith-Lovin, and James M Cook. Birds of a feather: Homophily in social networks.
Annual Review of Sociology, 27(1):415 444, 2001.
Shashank Pandit, Duen Horng Chau, Samuel Wang, and Christos Faloutsos. Netprobe: A fast and scalable system for fraud detection in online auction networks. In Proceedings of the 16th International Conference on World Wide Web, WWW '07, pp. 201-210. Association for Computing Machinery, 2007. ISBN 9781595936547.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, el al. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32 (NeurIPS), pp. 8026-8037, 8037. Curran Associates, Inc., 2019.
Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. Geom-GCN: Geometric graph convolutional networks. In International Conference on Learning Representations (ICLR), 2019.
Oleg Platonov, Denis Kuznedelev, Michael Diskin, Artem Babenko, and Liudmila Prokhorenkova. A critical look at the evaluation of gnns under helerophily: Are we really making progress? In International Conference on Learning Representations (ICLR), 2023.
Daniil Polykovskiy, Alexander Zhebrak, Benjamin Sanchez Lengeling, Sergey Golovanov, Oktai Tatanov, Stanislav Belyaev, Rauf Kurbanov, Alcksey Artamonov, Vladimir Aladinskiy, Mark Vesclov, et al. Molocular sets (MOSES): A benchmarking platform for molecular generation models. Frontiers in Pharmacology, 11: 565644, 2020.
12

---

Published in Transactions on Machine Learning Research (04/2025)
Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific Dala, 11))1177 2014.
David Rogers and Mathew Hahm. Fxtenneed-comeecivity fingerprints. Journal of Chemical Information and Modeling, 50(5):742 754, 2010.
Benodek Rozemberczki, Carl Allen, and Rik Sarkar. Multi-scale attributed node embedding. Journal of Compler Networks, (2))ee4,,,1, 2021.
Oleksandr Shehur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. Pitfalls of graph neural network evaluation. arXiv preprint ar 181 1ato:1811.0880000 1.05868, 2018.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: a flow- based autoregressive model for molecular graph generation. International Conference on Learning Representations (ICLR), 2019.
Jie Tang, Jimeng Sun, Chi Wang, and Zi Yang. Social influence analysis in large-scale networks. In Proceedings of the 15th ACM SIGKDD international conference Knowledge discovery and data mining, pp. 807 816, 2009.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. International Conference on Learning Representations (ICLR), 2018.
Yogesh Verma, Samuel Kaski, Markus Heinonen, and Vikas Garg. Modular flows: Differential molecular generation. In Advances in Neural Information Processing Systems 35 (NeurIPS), pp. 12409-12421. Curran Associates, Inc., 2022.
Yu Wang and Tyler Derr. Tree decomposed graph neural network. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM '21, pp. 2040 2049. Association for Computing Machinery, 2021.
Yuelin Wang, Kai Yi, Xinliang Liu, Yu Guang Wang, and Shi Jin. ACMP: Allen-Cahn message passing with attractive and repulsive forces for graph neural networks. In International Conference on Learning Representations (ICLR), 2023.
David Weininger, Arthur Weininger, and Joseph L Weininger. SMILES. 2. Algorithm for generation of unique SMILES notation. Journal of Chemical Information and Computer Sciences, (9(2):97 101, 1989.
Lirong Wu, Haitao Lin, Bozhen Hu, Cheng Tan, Zhangyang Gao, Zicheng Liu, and Stan Z Li. Beyond homophily and homogencity assumption: Relation-based frequency adaptive graph neural networks. IEEE Transactions on Newral Nelworks and Learning Systems, 2023.
Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In Proceedings of the 35th Interna- tional Conference on Machine Learning (ICML), volume 80 of Proceedings of Machine Learning Research, pp. 5453-50 -462. PMLR, 2018.
Keyulu Xu, Weihua Hu, Jure Teskovec, and Stefanie Jegelka. How powerful are graph neural networks? Tn International Conference on Learning Representations (ICLR), 2019.
Minkai Xu, Alexander S Powers, Ron O. Dror, Stefano Ermon, and Jure Leskovee. Geometric latent diffusion models for 3D molecule generation. In Proceedings of the 40th International Conference on Machine Learning (ICML), volume 202 of Proceedings of Machine Learning Research, pp. 18592-386118 PMLR, 2023.
Yujun Yan, Milad Hashemi, Kevin Swersky, Yaoqing Yang, and Danai Koutra. Two sides of the same coin: Heterophily and oversmoothing in graph convolutional neural networks. In JEEE International Conference Data Mining (ICDM), pp. 1287-1292. IEFE, 2022.
13

---

Published in Transactions on Machine Learning Research (04/2025)
Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In Proceedings of The 33rd International Conference on Machine Learning (TCML), volume 48 of Proceedings of Machine Learning Research, pp. 40 40-48. PMLR, 2016.
Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. In Advances in Neural Information Processing Systems 31 (NewrIPS), pp. 6410 6421. Curran Associales, Inc., 2018.
Chengxi Zang and Fei Wang. Moflow: An invertible flow model for generating molccular graphs. In Proceedings of the 26th ACM SIGKDD International Conference Knowledge Discovery & Dala Mining, pp. 617-621 626, 2020.
Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In Advances in Neural Information Processing Systems 33 (NeurIPS), pp. 7793 7804. Curran Associates, Inc., 2020.
Jiong Zhu, Ryan A Rossi, Anup Rao, Tung Mai, Nedim Lipka, Nesreen K Ahmed, and Danai Koutra. Graph neural networks with heterophily. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11168-11176, 2021.
Jiong Zhu, Junchen Jin, Donald Loveland, Michael T Schaub, and Danai Koutra. How does heterophily impact the robustness of graph neural networks? theoretical connections and practical implications. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2637-2647, 2022.
14

---

Published in Transactions on Machine Learning Research (04/2025)
Appendices
This appendix is organized as follows. App. A presents more about HetFlows: Prerequisite knowledge of normalizing flows and affine coupling layers, training process, generation process, loss function, details of MixMP GNNs in FetFlows, computational issues and reversibility proof of the model. App. illustrates related details of node classification experiment, including the experiment environment, data sets information and message passing details of GNNs used as base models. App. C provides the experiment environment, data sets, additional experiment results for molecule generation, property optimization algorithm with corresponding results together and model selection details. App. D summarizes and describes the metrics used in the molecular generation experiments.
A HetFlows Details
This section presents all the technical details HetFlows.
A.1 Prerequisites: Normalizing Flows with Affine Coupling Layers
Affine coupling layers (ACLs) introduce reversible transformations to normalizing flows, ensuring efficient computation of the log-determinant of the Jacobian (Kingma & Dhariwal, 2018). Typically, the affine coupling layer, denoted by (FMM contains a binary masking matrix M E 1) and a coupling function f which determines the affine transformation parameters. The input O X by masking, where 'Q' denotes the Hadamard (element-wise) product.
Rmxx split into
X1 M C X and X₂ (1 M) Here, X1 is the masked input that will undergo the transformation, and X₂ is the part that provides parameters for this transformation via the coupling function and stays invariant inside the ACLs. The output is the concatenation of the transformed part and the fixed part, as visualized in Fig. A5 given as:
(CL(
x (1- M)
99)
such that log S,T = The binary masking ensures that only part of the input is transformed, allowing the model to update certain features while fixing others, enabling the model's reversibility. The coupling functions of flow capture intricate data characteristics, into which we incorporate heterophily priors.
Normalizing flows offers a methodological approach to model distribution based on the change-of blange-of-vvarable variaaee law of probabilities. This is achieved by applying a chain of reversible and bijective transformations between trivial variables (like Gaussian) with target data variables and updating the transformation minimize the negative log-likelihood (Dinh et al., 2014). Given target distribution zo Pz: we initialize flows f = fr ................................ f1. The flow reach trivial varibales ZT N(u, o²) through a series of invertible functions: z, = 1), ........ T. The goal of normalizing flows is to minimize the negative log-likelinn likelihoodds (NLLs)
of the data:
O
C = - logg = log N(2T (4, 0²) - log det
(10)
2zo
ACL
Split muoking
X1 - Max
X₂ -11 M.X
coupling function f)x
CONCAT O X +
log S.
X
Figure A5: The affine coupling layer. The coupling is defined through coupling function f and binary masking matrix M (seen in Eq. (9)).
15

---

Published in Transactions on Machine Learning Research (04/2025)
The power of normalizing flows lies in their bijectiveness. This ensures that no information from the data is lost during these transformations. Thus, the transformed distributiin can be 'pulled back' to the original space using the inverse of the transformation functions, providing a bridge between the Gaussian and the intricate target distribution.
A.2 Training Process
Molecule graph G contains atom (node) features X Rxxx= and bond (edge) features E R.xax., The terms Ra and ns denote the types amount of atoms and bonds respectively. And (X); denotes the one-hot encoded type of the tth atom present in molecule G. Similarly, (E)ay denotes the one-hot encoding of the specific chemical bond wet the zth and jth aloms. Our model HetFlows maps the molecule G to embeddings ha and h2 that follow the Gaussian distributions:
ha Pa N(Haa oà), hB Ps = N(H,,
(11)
Bond flow The bond flow represented by fo = ACLE, .... ACL; consists of a series of ACLs with convolutional neural networks (CNNs) coupling function: ACL ACL (CNN,, MM E R i i= (ke) (0) are updated
1.2...., kb. where ky denotes the number of layers. Then bond embeddings hs h fo(h),
by layers:
h
ACT (n1. (n i= 1.2...,,
(12)
initialized from the bond tensor h, E
Heterophilous atom flow The atom flow fa contains hi affine coupling layers. Each layer consists of a masking matrix M and a GNN coupling function GNN
ACI (GNN,
1.2
(13)
where (Mi), 1 j=n()), and I' forig., hom., het.) indicate MP scheme as mentioned in Sec. 3.1. GNN is a MixMP GNN combining these three schemes as described at App. A.5. All GNNs in this context derive their graph topology (E, E) from the bond lensor E. The embeddings are initialized by the atom features: h(a) X, and undergo update through each affine coupling layer as follows:
( ACL; (h 1E). i= 1,1....., ha.
(14)
The final node embedding h, h/ (1.m) fa(h
Training target The training goal of HetFlows is to decrese the distance between the transformed variables (ha, hy} with target distribution Pa: Pb, as assumed Eq. (11). The loss function combines the NLLs from both the atom and bond flow: L = L. + La (details at App. A.4). Each NLL is given as shown in Eq. (10). The target of training is to search for model parameters to minimize the loss:
Sas, Sox arg min L fn.f
(15)
A.3 Generation Process
Given trained HetFlows model, with established atom flow fax and bond flow for. the procedure for generating molecules is described as follows (* denotes parameters fixed).
Start by randomly sampling embeddings ho Pa and hs ~ pb from a Gaussian distribution as expressed in Eq. (11).
Sampling Embeddings:
16

---

Published in Transactions on Machine Learning Research (04/2025)
2. Obtaining the Bond Tensor: The bond tensor E can be derived by applying the inverse of the bond flow Jix to the sampled embedding hg- This is given as E Jo+ (hs)-
3. Recovering Graph Topology: From the bond tensor E, the graph topology (E, E) can be deduced. This topology is essential for the node feature generalion at the next step.
Generating Node Fcatures: With the bond tensor in place, the generated node features can be produced by applying the inverse of the atom flow faa to the sampled alom embedding ha given by X Jas E).
Molecule Recovery: Finally, molecule, represented as G (X, E), can be reconstructed from random embeddings [ha, ho]-
The topology generation process (Steps 2-3) can be achieved by sampling the adjacency matrix from the real (data) distribution, which is denoted by ++true adj.
Reversibility of Fetws
To ensure that the molecular embeddings produced by HetFlows can be inverted back, it crucial understand the reversibility of the processes. A formal proof of reversibility of ACL blocks and HetFlows is provided in App. A.7.
A.4 Loss function
The loss function of the HetFlows is the NLLs of flows L La Lo consists of atom flows and bond flows. The details are listed as follows: The bond model loss La comes from the NLLs the Eq. (10)
PACT: (n
L. = - l (hr) - E log det
(16)
Similarly, the loss Ca for the atom flow can be constructed as:
La log (ha) Σ/= log det (1 [ )
(17)
A.5 The MixMP Graph Neural Network
In the code implementation, the MixMP GCN, GNN (r = forig. hom., het.)), consists of convolutional layers, batch normalization layers and a series of linear layers. The convolutional layer is the improved version of the graph convolutional layer of GCNs (Kipf & Welling, 2017), with three channels of MP. Each channel transfers messages between neighbours by certain preferences based on the homophily metoohilly/eeteropee hhetropphly
Given imput h E Raxn, and bond tensor E E puxnxm, assume the adjacency matrix A, E R*xn i= 1,..., ng. Here ny are the feature dimensions of the node and bond. Then the messages are scaling with the scaling malrix H., E BRnx,, and (Ha)is = 0gj.r. The homophily factors Que,y differ from channels indicated by 7
M(u, v).
if y = hom.
Query
1, 1 H(u,v),
if 7 orig. if y het.,
(18)
and H(u, v) + (h ( is the cosine similarity. The node embeddings are updated by the transferred messages in these triple channels, for each edge channel i,
h cat Horig. @ A,h, Hhom. @ A,h, Heet C A.) W..
(19)
for i i 1,..., no, where the (W, E R 18 Tizl are the model parameters, and cat denotes the concatenation operator, nout is the expected output dimension. Then the output of this convolutional layer is the sum of all edge types
h = Σ h E R nx
(20)
17

---

Published in Transactions on Machine Learning Research (04/2025)
In conclusion, given the input h E R.x*, and bond tensor E E R"xxx", the convolutional layer generates ouput
GNN (h E) = E R
(21)
A.6 Computational Considerations
For convenience on the calculation of the log-likelihodd, every transformation of variables needs the calculation of Jacobian matrix (i.e., OZ((+1) ()) So all the complicated modules (e.g., GNNs, MLPs) are all built inside the coupling structure (part of the input is updated by the scaling matrix S, and transformation matrix T depends on the other part of the input).
A.7 Proof of Reversibility of HetFlows
Both the atom and bond models of HetFlows rely on ACL blocks. As introduced in App. A.1, these blocks are inherently reversible. This means they can forward process the input to produce an output and can also take that output to revert it to the original imput without loss of information. Besides the use of ACL blocks, the operations used within the model primarily leverage simple concatenation or permutation. These operations are straight forward and do not alfect the overall reversibilityy of the processes. Given that the individual components (both atom and bond flows) are reversible and the operations performed on the data are straightforward, thus that HetFlows whole is reversible. Fromal proof of the model's reversibility is provided below.
A.7.1 Reversibility of the ACL
Claim The affine coupling layer (ACL) defined at App. A.1, the atom model fa and bond model fo defined App. A.2 are reversible. Set up Assume an ACL (Kingma & Dhariwal, 2018) defined in App. A.1 contains
coupling function f and masking matrix M {0, 1,mx", Given input X Rmxx, the output Y is calculated
== (CL( (X)
(22)
( M)
where log S,T f(X2), and X1, X2 are the split from input by masking:
x = MOX, X (1-
(23)
We seek to recover X from the M,M, and Y. Reversibility from output to input
Since M binary, we can get the following results.
M M = M,
(24)
(1- M) G (1-M) = (I-M),
(25)
M M - (1-M
(26)
and
X (M + (1-111)))) 0X
(27)
M x x
(28)
X + X
(29)
By splitting the output Y to Y,, Yg by masking matrix:
Yi MOY, Y = =1 (1-
(30)
18

---

Published in Transactions on Machine Learning Research (04/2025)
Combining with Eq. (22), we know
Y MCY
(31)
M X (1- M)
(32)
+T7)
(33)
and
Y= = (1
(34)
(1 M) (M O
((1-M)@X
(35)
= (1- (
+(1 M) 0 11-MM
(36)
(1- O X2.
(37)
Now the log S,T f(X2) Y2 are recovered by Y. Notice that
M es
(38)
MO ( + - 8S
(39)
=
(40)
-MOSO e
(41)
-
(42)
-OMOX
(43)
MOO
(44)
= X, (S), 0, Vi.j.
(45)
where '0' denotes element-wise division. Since we define as the exponential part of the output from the coupling function, the clements of S are all strictly positive. Then
x = (ACI(U.M) (Y)
X1+
(46)
-
M (Y -MG
where log S,T 1(X2) f(1 M Eq. (46) shows how the input recovered from the output, thus the ACL block is reversible.
A.7.2 Reversibility of the Bond Model
For the bond model fo ACI a... ACI,, and since each ACI 1= hy is reversible, we write 55" (ACL+) o...o (ACI. which the reverse function of fo-
A.7.3 Reversibility of the Atom Model
For the atom model Sa ACLA o... ACL,, and since each ACL,, i= ,i. is reversible, write f-1 = (ACL))-1 o...o (ACLg.) which the reverse function of fa-
B
Experiment Details: Node Classification
B.1
Hardware
All models this experiment are trained on Limux cluster equipped with NVIDIA V100 GPUs. The training time and memory requirement for single were (for all modes orig., hom., het., mix. and for all architectures):
19

---

Published in Transactions on Machine Learning Research (04/2025)
TEXAS: 10 mins, 2 GB
CORNELL: 10 mins, 2 GB
WISCONSIN: 10 mins, 2 GB
SQuirrel: mins, GB
CHAMELEON 10 mins, GB
CIreSEER: 10 mins, 2 GB
COMPUTERS: 20 mins, 16 GB
PUBMED: 10 mins, 4 GB
CORA: 10 mins, 2 GB
PHOтO: 15 mins, 8 GB
ROMAN-EMPIRE: 10 mins, 2GB
AMAZON-RATINGS: mins, 2GB
MINESWEEPER: 10 mins, 2GB
ToloKErs: 20 mins, 16GB
QUESTIONS: 15 mins, 8GB
B.2
Data Sets
There are 15 data sets selected to conduct comprehensive and discriminative node classification tasks. To examine patterns associated with varying levels data homophily, half of the data sets are chosen as having higher homophily, while the other half are more heterophilic. The data domains include citation networks (Yang al., 2016) (CORA, PUBMED, CITESEER), co-purchase graphs (Shchur el. al., 2018) (COMPUTERS, РHOто), hyperlink networks (Pei et al., 2019) (CORNELL, WISCONSIN, TEXAs), Wikipedia networks (Rozemberezki et al., 2021) (CHAMELEON, SQUIRREL), and heterophilous graph datasct (Platonov et al., 2023) (ROMAN-EMPIRE, AMAZON-RATINGS,, MINESWEEPER, TOLOKERS, QUESTIONS).
Statistical information The descriptive statistics all the data sets for node classification tasks in Sec. 4.1 displayed in Table A4. Here, Ngraphs: Nnodes and Nedges denote the total amounts of graphs, nodes and edges in the data set respectively. Dieat, Nelass denole the dimensions of the node features and labels. Ha, He and Hes denote the average node homophily (Pei et al., 2019), average edge homophily (Zhu et al., 2020) and average class insensitive edge homophily ratio (Lim et al., 2021) of the data set.
Data split rule For three heterophilic data sets (CORNELL, WISCONSIN and Texas), the data is split to train/ httn/validate/teee with fixed 10 seeds from GROM-GCN (Pei et al., 2019). For other datasets, the splitting is randomly controlled by the python torch package. Each result contains 10 random runs for data split and model initializations.
B.3 GNN Algorithm Details
There are four classic GNNs utilized as base models in the node classification task of Sec. 4.1. Here we provide these algorithms by showing the message- passing details of a single convolutional layer.
Graph Convolutional Networks (GCN) The most classic graph convolutional operator is proposed by Kipf& Welling (2017). Given the embeddings h of node v V at 4th layer, the message for node from node defined as
ml
Cu,v (())
(47)
(2)2
20

---

Published in Transactions on Machine Learning Research (04/2025)
where Cu,o denotes the weight of edge (u, v), d =1= E (v)(()) Ca,v denotes the weighted degree of node with self-loop. Then the node embeddings are updated by the message set {(mawll M(u) {u}}} collected from all its neighbours
h (k- ( T
muy
(k)
(48)
(CN(M)
where + denotes the model parameters.
Graph Attention Networks (GAT) Veličković et al. (2018) proposed the graph attention operator inspired by the transformer. Given the embeddings h of node e V at hth layer, the message for node from node u is defined as
au,o O (R)
(R)
if
m (k)
(49)
au O,h, hig
uu=
where O, O denotes the model parameters, and aueo denotes the attention value from node v to node
exp (TR(aa 1 + oaT
G Σ exp (LR(aa) b, + a, 0,))))
(50)
where LR denotes the Leaky Rectified Linear Unit. Then the node embeddings are updated by the neighbour message set
(k+1)
Σ
M
(51)
(
Graph Isomorphism Network (GIN) The graph isomorphism operator is created by Xu et al. (2019) based on Weisfeiler Lehman (WL) graph isomorphism test (Leman & Weisfeiler, 1968). Given the embeddings he of node v at 4th layer, the message for node w from node u the node embedding itself
il u +
m(k)
(k)
(52)
(1++)
=
where the controls the selg-weight. Then the node embeddings are updated by the neighbour message set through another neural network hg
1 [k+1) he Σ
mau
m
(53)
Table A4: The statistic information on the data sets for node classification experiment in Sec. 4.1
Ngraphs
N Moodes
Nedges
Dieat
Nelase
Hnn
He
Hes
TEXXS
183
325
1703
5
0.104
0.108
0.001
MINESWEEPER
10000
78804
7
2
0.683
0.683
0.009
ROMAN-EMPIRE
22662
65854
300
18
0.046
0.047
0.021
SQUIRREL
5201
217073
2089
0.219
0.224
0.026
CORNELL
183
298
1703
5
0.106
0.131
0.059
CHAMELEON
2277
36101
2325
0.249
0.235
0.063
QUESTIONS
48921
307080
301
2
0.898
0.840
0.079
WISCONSIN
515
1703
5
0.134
0.196
0.097
AMAZON-RATINGS
24192
186100
300
0.376
0.380
0. 0.127
TOLOKERS
11758
1038000
10
0.634
0.595
0. 0.180
CITESEER
3327
9101
3703
6
0.706
0.736
0.627
PUBMED
19717
88648
500
3
0.792
0.802
0.664
COMPUTERS
13752
491722
767
10
0.785
0.777
0.700
CORA
2708
10556
1433
7
0.825
0.810
0.766
PHOTO
7650
238162
745
0.836
0.827
0.772
21

---

Published in Transactions on Machine Learning Research (04/2025)
Table A5: Performance comparison between GraphSAGE+ with other heterophily-aware GNNs, the best result among the bascline modes is highlighted in bold.
Texas
Cornell 0.06
Wisconsin 0.10
CiteSeer 0.63
PubMed 0.66
Cora
Homophily Her
0.00
0.77
GraphSAGE+miix
79.745.9
75.114.0
81.5111
77.0g1.
89.3204
87 8:12
GraphSAGE
79.0112
71.411.2
64.816.1
78.2.0.
86.8_0.1
ACMP
86.215.0
85.417.0
86.14
75.0s1.
78.9-1.
84.9:0.8
H2GCN
85.943.5
80.2+4.7
87.5+1a
80.0-07
87.8-0a
S7.5+06
GPRGNN
92.949.
91.4+0.7
93.842.4
67. 67.6200
85.120.1
79.520.4
Geom-GGG GCN
67.6g8/4
60 80AA
64.1g8/4
78.0=M/4
90.02N:4
85.344/4
ACM-GCN
94.9:2
94.8138
95.8120
81.7210
90.7:0.5
88.6:12
The he is often defined a simple MLP.
GraphSAGE Hamilton et al. (2017) highlights the sample and aggregation approach their GraphSAGE
operator. Given the embeddings he of node V at hth layer, the message for node v from node is the node embedding itself
m = vh (k)
(54)
Then the node embeddings are updated as
h (k+1) θ mm.. um + O2 (w))
Σ oEN(u)
Mayy
(55)
where Oj, O2 are the model parameters.
B.4 Comparison with other heterophily-aware GNN baselines
We select -rap compare with baselines on node classification tasks over data sets in Table A5. The baseline models selected the base model and other heterophily-aware GNN: ACMP (Wang et al., 2023), H2GCN (Zhu et al., 2021), GPRGNN (Chien et al., 2021), Geom-GCN (Pei et al., 2019) and ACM-GCN (Luan et al., 2022). The GraphSAGF-+mix. are generally better than its original model: GraphSAGE. It shows the representation enhancement through our heterophilous message- saasaag-passingg scheme. Here ACM-GCN is the best algorithm over all algorithms such comparison. The N/A means the reported results are not available from the baseline papers.
C Experiment Details: Molecule Generation
C.1 Hardware
All models in this experiment are trained on cluster equipped with NVIDIA A100 GPUs. The training time for a single model was 10 h (QM9) and 90 h (ZINC-250K).
The training time and memory requirement for single models were (for all modes orig., hom.or het.and for all base models)
QM9: 10 h, 8 GB
ZINC-250K: 90 h, 16 GB
C.2
Data sets
Two classic molecule data sets (QM9 and ZINC-250K), which are widely used in academia, are chosen for the molecular generation task. The QM9 data set (Ramakrishnan et al., 2014) comprises 1334 stable small organic molecules composed of atoms from the set {C, H, O, N, F}. The ZINC-250K (Irwin et al., 2012) data contains 250k like molecules, each with up to 38 atoms of 9 different types.
Data split. In this experiment, all data sets split with ratio train/test 80/20%.
22

---

Published in Transactions on Machine Learning Research (04/2025)
aud 0.10
OH
frm
0:36
0:18
Figure A6: Structured latent-space exploration (ZINC-250K). Example of nearest neighbour search the latent space with the seed molecules on the left and neighbours with the Tanimoto similarity (1 given for each molecule. For results on QM9, see Fig. 4 in the main paper.
0)
aq.
Grept07
Holton
5.0
0.2
0.8
200
600
logP
8A
QED
Molecular weight
Figure A7: Chemoinformatics statistics for data (ZINC-250K) and generated molccules from HctFlows (ours), MoFlow, and GraphDF. Histograms for the Octanol-water partition coefficient (logP), synthetic accessibility score (SA), quantitative estimation of drug- line-sikeness (QED), and molecular weight.
Table A6: Full benchmark metrics for random generation using QM9 (reporting tean
Validity
Uniqueness
Novelty t
SNN f
Frag 1
Seaf 1
IntDiv;
Data (QM9)
1.0020.00
1.0020.00
0.62=0.02
0.54_6.00
0.94
0.76
0.92:0.00
GraphDF
1.00=0:00
0.9820.00
0.35_0.00
0.6120.01
0.0920.07
0.87=0.00
MoFlow
0.9520.01
1. 1..000.00
0.96-0.01
0.32-0.00
0.6020.05
0.04g0.08
0.92-0.00
HetPlow (Ours)
0.92+0.01
0.9910.00
0.92-6.1
0.34-0.00
0.80=0002
0.04
0.91
Mollow--true adj.
1.00:0.00
1.00-0.00
0.85.0.01
0.38-0.00
0.70:0.0s
0.3120.08
0.92:00.0
HetFlow+true adj.
1.00
1. 1.00=0:00
0.74-0.01
0.4320.00
0.85g0.02
0.52g0.0G
0.9220.00
EDM
0.92+0.01
1.00-0.0
0.56-0.2
0.48
0.92=0.01
0.65
0.92-000
IntDiva
Filters f
FCD
AlogP 4
ASA 4
AQED
A Weight
Data (QM9)
0.90+0.00
0.6420.02
0.40-002
0.04-001
0.03g0.01
0.00g0.00
0.32:0.05
GraphDP
0.86+
0.60-0.02
10.76+0.21
0.16-
0.27+0.02
0.05+0.00
19.72-0.4
MoFlow
0.3540
7.55-0.23
0.40-002
0.41:0.02
0.01:0.00
3.75-00.s
HetFlow (Ours)
0.90:0.00
U. 0.62220.2
4.04_0.24
0.11g0.02
0.34g0.01
0.0330000
4.40=0.11
MoFlow-+true ad).
0.90
0. 0.
4,45g001
0. 0.65-002
0.50g0.02
0.04g..00
0.74-004
HetPlow-ttrue adj.
0.9020.00
0. 0.6220.0L
1.46_0.00
0.26-0.02
0.36g0.02
0.0220.00
1.12:0.18
EDM
0.90=
0.61
0. 0.9620.a
0. 0.16-0044
0.15=0.4
0. 0.01
1.87:036
C.3
Further Results
Latent-space exploration We provide further results for struetured latent-space exploration. Example explorations for QM9 and ZINC-250K are shown in Fig. 4 and Fig. A6.
Overall 14 metrics tables We include full listings all 14 metrics (description of metrics in App. D) considered in the random generation tasks for QM9 and ZINC-250K. The values are listed in Tables A6 and A7, respcctively.
The reconstruction example For better understanding, we provide reconstruction examples on QM9 from intermediate layers in Fig. A8.
23

---

Published in Transactions on Machine Learning Research (04/2025)
Table A7: Full benchmark metrics for random generation using ZINC-250K (reporting mean
Validity f
Uniqueness f
Novelty f
SNN t
Frag 1
f
IntDivi
Data (XNC-250Kk)
1.0020.00
1.0040.00
0.02:0.00
0.51 20.00
1.0020.00
0.28g0.02
0.87
GraphDF
1.00:0.00
1.0000.00
0.23 0.23
0.35
0.00g0.00
0.8820.00
MoFlow
0.8010.01
1.00:0.00
1.00:0.00
0.27
0.79:0.01
0.01g0.00
0.8810.00
HetFloww (Ouns)
0.87
1.00
1.00=0.00
0.26
0.77
0.01
0.88
MoFlow trrue adj.
0.9410.01
1.00:0.00
1.0000.00
g.03
0.8920.01
0.07g0.02
0.87
HetFlow true adj.
0. 0.9320.01
1.00:0.00
1.00
0.34
0. 0.9120.01
0. 0.10
0.87
Int f
Filters 1
FCD 4
AlogI' 4
ASA
AQED
A Weight $
Data (2NC-2500k)
0.8610.00
0.3910.01
1.4420.01
g.05
0,03g0.01
0.0120.00
2.18g0.39
GraphDF
0.87+0.00
0.54+0.00
30.30
1.28:0.00
1.70=0.03
0.30=0.00
+1.02
MoFlow
U. 0.000
0,51
23 20.33
0. 0.15
0.8220.03
0.2520.01
56.71:2.64
HotFlow (Ours)
0.87:1000
0.5110.01
23.72:0.19
0.50
0.99
0.25
51.90:1.6
MoFlow + true adj. HetFlos Flo true adj.
0.8611000
0,6710.02
8. 8.2120.22
0. 0.64
0,5420.00
0.04
4.251
0.8612.00
0.7520.01
8.24-0.17
0.82
0.41
0.04:0.00
5.05g0077
NM
NH
0H
HD
101
18
OH
N0
M
OH
Layer 6=0
Layer i=4
Layer 8=8
Layer x= 10
Layer i- 16
Layer 2=
Layer i= 26
Figure A8: Step-by-stop generation (QM9). Snapshots of reconstructed molocules when fixing the bond model and collecting node embeddings of the intermediale layers i.
Homophily distribution of generated molecules Additionally, we also visualize the node homophily for both QM9 and ZINC-250K together with the estimated node homophily histograms (see Fig. A9) from the generation outputs from the different models. The adjacency matrix generation strategies have a minor inffuence on the homophily distribution of generated molecules, it shows these statistics mostly rely the alom model but not the bond model.
Ablation study The random generation of HetFlows with/without, parameters sharing in the MixMP GNNs are shown in Table A8 and Table A9,
Table A8: Ablation study of MixMP GNN parameter sharing on QM9
FCD J
Validity
Novelty
SNN f
Frag
Scaf 1
IntDiv, f
Data (QM9)
0. 0.10g0.2
1.00g0.00
0.6240.32
0.54-000
0.94
0.7629.3
0.92:0.00
HotFlow+trro adj-
1 0.
1.00+ 1.0020.00
0.7440.0
0.43
0.86ea.0
0.52+4.05
0. 0.
Het FlowFootrre +true tshare para.
1.66
0. 0.77+0.01
0. 0.4220.00
8..0.022
0.4844.55
0. 0.922.0.00
24

---

Published in Transactions on Machine Learning Research (04/2025)
gmd HisFloo-trrue adj
hecRoowrtrue ad
hethoww
heshlow
Graphör
Graph07
NoFlow
MoFlow
Murlow-iine tnue
Moffion--irw
EDM
0.0
0.2
0.4
2.6
1.0
0.0
0.2
0.4
0.6
0.8
10
Qu4
ZINC-25UK
Figure A9: Node homophily distribution of generated molecules.
Table A9: Ablation study of MixMP GNN parameter sharing ZINC-250K
FCD
Validity
Novelty
SNN f
Frag
Scaf f
IutDivi f
Data (ZIN0-250k)
1.44g001
1.002a.00
0.0240.00
0.51-000
1.00g0.00
0.284aa2
0.87=0.0
Fet adj-
8.24+017
0.93+u.0
1.00+0.00
0.34+0.0
0.01
0.10+a.03
0.87+0.00
HetFlow +shaee
8. 11
0.93+0001
1.00=0.00
0.32=0.0
0.93+0.01
0.05
0 0.8740.00
Table A10: Performance on molecule property optimization in terms of the best QED scores, scores taken from the corresponding papers (JTVAE score from Luo et al., 2021; Verma et al., 2022).
Method
1st
2ud
3rd
Data (2INC-250k)
0.948
0.948
0.948
JTVAE
0.925
0.911
0.910
GCPN
0.917
0.946
GraphAP
0.948
0.948
0.947
GraphDF
MoFlow
0.948
0.948
0.948
ModFlow
0.948
0.948
0.945
HetFlows
C.4 Property Optimization
In the property optimization task, models show their capability to find novel molecules that optimize specific chemical properties not present in the training data set: critical component for drug discovery. For our study, we focused maximizing the QED property. We trained HetFlows on ZINC-250K and evaluated its performance against other state-of-the-art models (Verma et al., 2022; Luo et al., 2021; Zang & Wang, 2020; Shi et al., 2019; Jin et al., 2018; You et al., 2018). The results, given in Table A10, show that the top three novel molecule candidales identified by Hel (not part of the ZINC-250K data set), exhibit QED values on par with those from ZINC-250K or other state-of-the-ar methods.
Algorithm Given a pro-trained HetFlows f. and training set D contains molccule and property label pairs [G. y). Now we introduce an extra simple MLP which maps the molecular embeddings f(G) into the predicted property
yp g=
(56)
it is trained on the dataset to be goe by optimizing the parameters:
arg min MSEloss( (f
(57)
(G.jCD
25

---

Published in Transactions on Machine Learning Research (04/2025)
Then find molecule candidates [G with top-k properties in the data set D are chosen. New embeddings are explored by optimizing the predict label by Do* starting from these candidates:
dgo* (hi.g-1) hij-b j I,
hij 8h
(58)
h4 f(G:),
i I.....
where 8 denotes the search step length, and N is the number of iterations. These embeddings could be recovered to be molecule set:
D' -f (f (hig))= j=
(59)
Finally, D'DD gives the novel molecule sets with related high target properties.
Gencration results In our experiments, the 9o is a simple 3-layer MLP with 16 hidden nodes, the datasot D ZINC-250K, and larget property QED. And D' provides 17 molecules with QED score 0.948. The Top-3 QED score and molecular SMILES are listed below:
Cuaee(((-DNNaaccccccoooooooooooo QED 0.947936,
2. CctremelN(/--ppprrrooooooCoCoooo QED 0.947505,
3. CCmmCCCccaarc-00oNccccuuCcccccae QED 0.947317.
Baselines The baselines scores of GCPN (You et al., 2018), GraphAF (Shi et al., 2019), GraphDF (Luo et al., 2021), MoFlow (Zang & Wang, 2020) and ModFlow (Verma et al., 2022) are acquired from the corresponding papers. The score of JTVAE (Jin et al., 2018) acquired from Zang & Wang (2020); Verma et al. (2022).
C.5 Model selection
The ranges of HetFlows hyperparameters are listed as follows:
The residual connection of ACL coupling function: (rue, False]
The parameter sharing mode of In Ma; In ps: [0, 1, 2]. Here Pas Mb are the variance of the embeddings distribution No in Eq. (7).
mode means the distribution variance of both node and edge embeddings are fixed to be 1: In Pa = In μbb = 0.
mode 1 means the distribution variance of both node and edge embeddings share one parameter:
In Pa In pb. it could be optimized during the training process.
-mode 2 means the distribution variance of both node and edge embeddings separate
parameters: In Has In μb, both of them could be optimized the during training process.
The best-performing model is selected using the FCD score as suggested in Polykovskiy al. (2020).
The HetFlows reported in the paper is selected with hyperparameters:
The residual connection of ACL coupling function: False
The parameter sharing mode of In Pa; In pa: 1.
26

---

Published in Transactions on Machine Learning Research (04/2025)
D Description of Chemoinformatics Metrics
For benchmarking, model selection, comparison, and explorative analysis, we use the following 14 metrics. The metrics are presented in detail in the work by Polykovskiy et al. (2020) that introduced the MOSES benchmarking platform. The metries calculation makes heavy use of the RDKit open-source cheminformatics software (https://war htttps:/wwwwrrkktt... wekit.dkkr........ We briefty summarize the metrics below.
Sanity check metrics
Validity Fraction (in [o.1]] 1]) of the molecules that produce valid SMILES representations. This is a sanity check for how well the model captures explicit chemical constraints such as proper valence. Higher values are better as low value can indicate that the model does not properly capture chemical structure. We report numbers without post hoc validity corrections.
Uniqueness Fraction (in [0, 1J) of the molecules that are unique. This is sanily check based on the SMILES string representation of the generated molecules. Higher values are better low value can indicate the model has collapsed and produces only few typical molecules.
3. Novelty Fraction (in [0, 11) of the generated molecules that are not present in the training set. Higher values are better as a low value can indicate overfitting to the training data set.
Summary statistics
4. Similarity to nearest neighbour (SNN) The average Tanimoto similarity (Jaccard coefficient) in [o, 1] between the generated molecules and their nearest neighbour in the reference data set. Higher is better: If the generated molecules are far from the reference set, similarity to the nearest neighbour will be low.
Fragment similarity (Frag) Measures similarity (in [0, 1]) of distributions of BRICS fragments (substructures) in the generated set vs. the original data set. If molecules in the two sets share many of the same fragments in similar proportions, the Frag metric will be close to 1 (higher better).
6. Scaffold similarity (Scaf) Measures similarity (in [0, 1]) of distributions of (molecule ring structures, linker fragments, and carbonyl groups) in the generated set vs. the original data set. This metric is calculated similarly to the Fragment similarity metric by counting substructure presence in the data, and they can be high even if the data sets do not contain the same molecules.
scaffolds
7. Internal diversity (IntDiv,) Measure (in [0, 1J) of the chemical diversity within the generated set of molecules. Higher values are better and signal higher diversity in the generated set of molecules. Low values can signal mode collapse.
Internal diversity (IntDivz) Measure (in [o, 1]) of the chemical diversity within the generated sot of molecules. The interpretation is similar to IntDiv, but with stronger penalization of the Tanimoto similarity in calculating the diversity.
9. Filters This metric is specific to the MOSES benchmarking platrofm (see Polykovskiy et al., 2020). It gives the fraction (in [0, 1]) of generated molecules that pass filters applicd during data set construction. In practice, these filters may filter out chemically valid molecules with fragments that are not of interest in the MOSES data set (filtered with medicinal chemistry filters). Thus, this metric not of primary interest to us but gives a view on the match with the MOSES data set.
10. Fréchct ChemNet distance (FCD) Analogous to the Frechét Inception Distance (FID) used in image generation, FCD compares feature distributions of real and generated molecules using a pre-trained model (ChemNet). Lower values are better.
27

---

Published in Transactions on Machine Learning Research (04/2025)
Descriptive distributions
11. Octanol-water partition coefficient (logP) A logarithmic measure of the relationship weew lipophilicity (fat solubility) and hydrophilicity (water solubility) of a set of molecules. For large values a substance is more soluble in fat-like solvents such as n-octanol, and for small values more soluble in water. We report both histograms of logP and summary statistic in terms of the Wasserstein distance between the generated and reference distributions (smaller better).
12. Synthetic accessibility score (SA) A metric that estimates how easily a chemical molecule can be synthesized. It provides quantitative value indicating the relative difficulty or ease of synthesizing a molocule, with lower SA score suggesting that molccule more casily synthesized, and higher score suggesting greater complexity or difficully. We report both histograms of SA and summary statistic in terms of the Wasserstein distance between the generated and reference distributions (smaller better).
13. Quantitative estimation of drug-likeness (QED) A metric designed provide a quantitative measure of how 'drug-like' molecule is. It essentially refers to the likelihood that molecule possesses properties consistent with most known drugs, estimated based variety of molecular descriptors. We report both histograms of QED and summary statistic in terms of the Wasserstein distance between the generated and reference distributions (smaller better).
14. Molecular weight (Weight) The sum of atomic weights molecule. We report both histograms of molecular weights and summary statistic in terms of the Wasserstein distance between the generated and reference distributions (smaller better).
28


Reviews:
[{"review_id": "BDAkeVRx2z", "reviewer": "Reviewer_Fdni", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper's main strength is that it proposes a general modification to the GNN, that can customise it either for a homophilic or a heterophilic use case, without introducing additional parameters or significant additional computation. The writing is generally clear, and the method is tested on different GNN x dataset combinations for node classification, as well as for graph generation.\n\nThe main weakness is that it is not clear how the findings can be used by the community -- the heterophilic datasets used are known to have some limitations (see [1], where also new datasets were proposed), the method is not compared with other approaches for tackling heterophily for GNNs, and the results don't have a clear suggestion on which the version of the module should be used (heterophilic, homophilic, original) in relation to the homophily of the dataset. Additional weaknesses are related to clarity and providing additional explanations, which I will describe in the next section.\n\n[1]: Platonov, Oleg, et al. \"A critical look at the evaluation of GNNs under heterophily: Are we really making progress?.\" arXiv preprint arXiv:2302.11640 (2023).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a new method for considering homophily and heterophily in message passing. In particular, the method proposes scaling the messages of each GNN layer by a function that depends on a global term gamma (corresponding to homophily or heterophily of the graph), and an edge-dependent term H(u,v) which computes the nodes' cosine similarity. \n\nThis method is applied to 4 GNN models (GCN, GAT, GIN, GraphSAGE) on a range of node classification benchmarks that vary in how homophilic they are based on their node homophily metric. In each instance, the original GNN is compared to its homophilic version (where all messages are scaled by the edge similarity) and its heterophilic version (where all messages are scaled by 1-edge similarity). In most combinations (GNN x dataset), either the homophilic or heterophilic version matched or outperformed the original GNN, with the homophilic version being more advantageous on homophilic datasets and similarly for the heterophilic; however, GAT and GIN provide exceptions to this intuition.\n\nThe method is also applied in combination with MoFlow, a normalizing flow model for molecular graphs. This model has an atom flow and a bond flow,  and it modifies the message passing in the same way as for the node classification GNNs. The model is tested on QM9 and ZINC, and compared with the original MoFlow version and GraphDF, matching their performance or exceeding, especially when sampling the adjacency matrix from the real data distribution, instead of using the bond flow.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "Adjustments that are critical to securing my recommendation for acceptance:\n\nI think the proposed method needs a more thorough integration of the findings from previous works addressing heterophily for GNNs. In particular, the method should be compared with other methods designed for heterophily (e.g. [2] or other related work). Similarly, previous papers have found limitations with some of the used datasets for evaluating heterophily (the [1] mentioned above) -- evaluating on the proposed benchmark could be a better solution. Lastly, both [1] and [2] mention the drawbacks of node homophily, and consider other metrics (e.g. adjusted homophily). As it is unclear how node homophily relates to the homophilic and the heterophilic versions of the GNNs proposed, it is possible that other metrics would provide a clearer picture on which module the community should use depending on how the dataset is characterized. \n\nAdjustments that, in my view, would strengthen the work:\n\nThere are three types of clarifications that could improve the readability of the paper:\n- Gamma: Gamma is a factor that is used for scaling each message, at each layer, but, if I understand correctly, it is treated as a hyper-parameter. However, the ablation study mentions that \"the three channels could share parameters\", from which I understand that all three versions (original, homophilic and heterophilic) are used together. It would be useful to clarify when gamma is set (as well as what other alternatives could be), and if different values of gamma are combined, how is the combination computed.\n- MoFlow: it would be useful to mention why MoFlow was chosen and if the validity correction used in the original MoFlow paper was used here as well.\n- 'true adjacency': it would be easier to follow if this was introduced before the results, and also if it was explained how the bond flow was used in this case.\n\n[2] Luan, Sitao, et al. \"Revisiting heterophily for graph neural networks.\" Advances in neural information processing systems 35 (2022): 1362-1375.", "broader_impact_concerns": "I do not have any concerns to raise on the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "iyTdBU48mF", "reviewer": "Reviewer_ALAZ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- a very simple scheme that can be adapted to any MP architecture\n- experiments on graph generation, which is quite original in this literature\n\nWeaknesses:\n- although I like the simplicity of the idea, I think it could be pushed a lot further. For instance, it seems the authors introduces three distinct modes, which are tested completely separately? (Unless I am mistaken?) Why not propagate all three channels, then let the GNN learn how to combine them? This would mitigate the need to know \"in advance\" if the dataset is heterophilic of homophilic.\n- In the same fashion, there are probably other type of \"similarity\" notions that could be tested for the structured MP\n- many \"heterophilic\" strategies are mentioned in the intro, but are not tested in the experiments, where the proposed scheme are only tested against the vanilla GNN versions. It could be useful to test a few of them.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors propose a novel message-passing scheme for heterophilic graphs. Given any base MP scheme, the authors produce two new \"modes\", homophilic or heterophilic, simply by scaling all messages by the cosine similarity (or one minus the cosine similarity) of the representations of the nodes in the previous layer. They test these different \"modes\" on four classical GNN architectures, and on a normalizing flow architecture for molecular generation, where they report improved result.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See above: more general architecture, experiments against other heterophilic GNNs", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DPF0yvkmET", "reviewer": "Reviewer_6iA2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n* I appreciated the motivation of the authors to enable their heterophily-aware GNN to applications beyond node classification, e.g., for the proposed molecular generation application.\n* Connected to the first point, the fact that the authors evaluate both in a predictive and generative setting increases the potential interest in this work and allows for a more rigorous evaluation of the proposed method.\n* The node classification setting is rigorously evaluated, using a high number of random seeds and data splits, as well as a fair model comparison.\n\n## Weaknesses\n### **W1**: The work is poorly motivated.\n\nThe main contribution of this work is a novel variant for message-passing to mitigate oversmoothing. However, oversmoothing is barely introduced and more importantly, existing approaches to mitigate oversmoothing are not discussed in detail, neither are reasons why standard message-passing leads to oversmoothing. As a result, after reading introduction, related work and Section 3.1. and 3.2., it does not become clear what concrete issues persist in existing approaches that warrant the novel message-passing approach proposed by the authors. For example, at the end of 3.1., the authors write\n\n> However, in practice, a naïve aggregation strategy typically mixes messages, especially in heterophilic areas, leading to the ‘oversmoothing’ problem\n\nfollowed by citations of various previous works. Here, the authors do not disucss *why* simply mixing homophilic and heterophilic messages leads to oversmoothing. As a result, it becomes difficult to understand why the new message-passing approach, presented in Section 3.2., overcomes these limitations and prevents oversmoothing.\n\nAnother example is the following excerpt from Section 2, when discussing other heterophily-aware GNNs:\n\n> These methods typically require additional parameters or specialized structures to achieve their effects. However, we hope to capture the effects with minimal modifications (no additional parameters), making the solution flexible and widely applicable.\n\nIt is not clear to me why these additional parameters hinder the application of the mentioned methods to e.g., molecular generation, such as the approach presented by the authors, nor why they make the methods less flexible.\n\n### **W2**: Lack of clear empirical evidence for effectiveness of proposed approach\nThe results in Table 1 do not convince me that the proposed method offers real benefits over the base GNN models. For example, for the GCN, GCN+het. wins on Cornell but is on par or looses to the base GCN on all other heterophilic datasets, despite the fact that we know that these datasets are heterophilic. In addition, the GCN+het. looses to GCN+hom. on Chameleon, meaning that the homophilic variant wins over the heterophilic variant on a heterphilic dataset. The inverse case also exists, where GraphSAGE+het. beats GraphSAGE+hom. on CoAuthorCS, and GIN+het. beats GIN+hom. on Computers, CoAuthorCS and Photo. As such, it appears to me that there is no *clear* signal that the heterophilic MP improves performance on heterophilic datasets and that the homophilic MP improves performance on the homophilic datasets.\n\nThis also somewhat weakens the claim made by the authors that\n\n> for the 6 heterophilic data sets, our MP schemes perform as well or better then the baseline (in 24 out of 24 cases)\n\nThis statement is true, yet only if one considers both heterophilic and homophilic MP jointly. Just looking at the performance of the heterophilic MP compared to the two other models on these datasets, the heterophilic MP is only on-par or better on 17/24 cases and beats both other models only in a *single* case: Cornell with GCN.\n\nTo summarize, my concern with the benchmark results is less with the performance of the homophilic MP and heterophilic MP per se, but with the fact that the heterophilic or homophilic nature of a particular dataset does not seem to be related all too well to whether the heterophilic or homophilic MP offers improvements.\n\n### **W3**: Oversmoothing is never actually evaluated\nWhile mitigating oversmoothing is a central motivation of this work, it is never actually measured on any of the experiments. Is oversmoothing even a concern on the evaluated node classification datasets? And if so, do the homophilic or heterophilic MPs actually mitigate it? Such an analysis could perhaps also shed light on the results mentioned in **W2**, where the homophilic MP outperforms heterophilic MP on heterophilic datasets and vice versa.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this work, the authors propose two new message-passing (MP) approaches, one designed for homophilic and one designed for heterophilic tasks, augmenting the standard message-passing on the original graphs that takes all neighbors of a node into account. Importantly, these MP approaches do not require access to the label distribution of the underlying tasks, as homophily and heterophily are incorporated based on the similarity of node representation instead. As a result, the MP approaches are applicable to applications where no label information is available, e.g., in the case of private or missing labels or in generative modeling applications. The MP approaches are evaluated on homophilic and heterophilic node classification tasks, as well as on molecular generation benchmarks.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* A concrete and detailed explanation of (a) why existing (heterophily-aware) do not effectively mitigate oversmoothing and (b) how these issues are overcome by the proposed approach. In addition, a concrete and detailed explanation how existing heterophily-aware GNNs lack the flexibility and applicability of the proposed approach.  *This is critical to my recommendation*.\n\n* A more nuanced discussion of the empirical results on node classification that reflects the fact that only rarely do the proposed homophilic and heterophilic MPs show clear benefits on homophilic and heterophilic tasks, respectively. *This is critical to my recommendation*.\n\n* I think that an empirical analysis on whether oversmoothing occurs in the base GNNs and whether it is effectively mitigated by the proposed homophilic and heterophilic MPs would strengthen the work but is not critical to my recommendation.\n\nIn case the authors feel that I have misunderstood parts of their work, I am happy to discuss the weaknesses above.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: After a successful rebuttal, the reviewers unanimously agree that the paper has sufficiently progressed and is in a good shape for acceptance into TMLR now. I concur with the reviewers -- well done!
Decision: Accept (poster)