Learning Graph Structure from Convolutional Mixtures

1 INTRODUCTION . Inferring graphs from data to uncover latent complex information structure is a timely challenge for geometric deep learning ( Bronstein et al. , 2017 ) and graph signal processing ( Ortega et al. , 2018 ) . But it is also an opportunity , since network topology inference advances ( Dong et al. , 2019 ; Mateos et al. , 2019 ) could facilitate adoption of graph neural networks ( GNNs ) even when no input graph is available ( Hamilton , 2020 ) . The problem is also relevant when a given graph is too noisy or perturbed beyond what stable ( possibly pre-trained ) GNN architectures can effectively handle ( Gama et al. , 2020a ) . Early empirical evidence suggests that even when a graph is available , the structure could be further optimized for a downstream task ( Kazi et al. , 2020 ; Feizi et al. , 2013 ) , or else sparsified to boost computational efficiency and model interpretability ( Spielman & Srivastava , 2011 ) . In this paper , we posit a convolutional model relating observed and latent undirected graphs and formulate the graph learning task as a supervised network inverse problem ; see Section 2 for a formal problem statement . This fairly general model is motivated by various practical domains outlined in Section 3 , such as identifying the structure of network diffusion processes ( Segarra et al. , 2017 ; Pasdeloup et al. , 2018 ) , as well as network deconvolution and denoising ( Feizi et al. , 2013 ) . We propose a parameterized neural network model , termed graph deconvolution network ( GDN ) , which we train in a supervised fashion to learn the distribution of latent graphs . The architecture is derived from the principle of algorithm unrolling used to learn fast approximate solutions to inverse problems ( Gregor & LeCun , 2010 ; Sprechmann et al. , 2015 ; Monga et al. , 2021 ) , an idea that is yet to be explored in the context of graph structure identification . Since layers directly operate on , combine , and refine graph objects ( instead of nodal features ) , GDNs are inherently inductive and can generalize to graphs of different size . This allows the transfer of learning on small graphs to unseen larger graphs , which has significant implications in domains like social networks and molecular biology ( Yehudai et al. , 2021 ) . Our experiments demonstrate that GDNs are versatile to accommodate link prediction or edge-weight regression aspects of learning the graph structure , and achieve superior performance over various competing alternatives . Building on recent models of functional activity in the brain as a diffusion process over the underlying anatomical pathways ( Abdelnour et al. , 2014 ; Liang & Wang , 2017 ) , we show the applicability of GDNs to infer brain structural connectivity from functional networks obtained from the Human Connectome Project-Young Adult ( HCP-YA ) dataset . We also use GDNs to predict Facebook ties from user co-location data , outperforming relevant baselines . Related work . Network topology inference has a long history in statistics ( Dempster , 1972 ) , with noteworthy contributions for probabilistic graphical model selection ; see e.g . ( Kolaczyk , 2009 ; Friedman et al. , 2008 ; Drton & Maathuis , 2017 ) . Recent advances were propelled by graph signal processing insights through the lens of signal representation ( Dong et al. , 2019 ; Mateos et al. , 2019 ) , exploiting models of network diffusion ( Daneshmand et al. , 2014 ) , or else leveraging cardinal properties of network data such as smoothness ( Dong et al. , 2016 ; Kalofolias , 2016 ) and graph stationarity ( Segarra et al. , 2017 ; Pasdeloup et al. , 2018 ) . These works formulate ( convex ) optimization problems one has to solve for different graphs , and can lack robustness to signal model misspecifications . Scalability is an issue for the spectral-based network deconvolution approaches in ( Segarra et al. , 2017 ; Feizi et al. , 2013 ) , that require computationally-expensive eigendecompositions of the input graph for each problem instance . Moreover , none of these methods advocate a supervised learning paradigm to learn distributions over adjacency matrices . When it comes to this latter objective , deep generative models ( Liao et al. , 2019 ; Wang et al. , 2018 ; Li et al. , 2018 ) are typically trained in an unsupervised fashion , with the different goal of sampling from the learnt distribution . Most of these approaches learn node embeddings and are inherently transductive . Recently , so-termed latent graph learning has been shown effective in obtaining better task-driven representations of relational data for machine learning ( ML ) applications ( Wang et al. , 2019 ; Kazi et al. , 2020 ; Veličković et al. , 2020 ) , or to learn interactions among coupled dynamical systems ( Kipf et al. , 2018 ) . Summary of contributions . We introduce GDNs , a supervised learning model capable of recovering latent graph structure from observations of its convolutional mixtures , i.e. , related graphs containing spurious , indirect relationships . Our experiments on synthetic and real datasets demonstrate the effectiveness of GDNs for said task . They also showcase the model ’ s versatility to incorporate domain-specific topology information about the sought graphs . On synthetic data , GDNs outperform comparable methods on link prediction and edge-weight regression tasks across different randomgraph ensembles , while incurring a markedly lower ( post-training ) computational cost and inference time . GDNs are inductive and learnt models transfer across graphs of different sizes . We verify they exhibit minimal performance degradation even when tested on graphs 60× larger . Finally , using GDNs we propose a novel ML pipeline to learn whole brain structural connectivity ( SC ) from functional connectivity ( FC ) , a challenging and timely problem in network neuroscience . Results on the high-quality HCP-YA imaging dataset show that GDN performs well on specific brain subnetworks that are known to be relatively less correlated with the corresponding FC due to ageing-related effects – a testament to the model ’ s robustness and expressive power . Overall , results here support the promising prospect of using graph representation learning to integrate brain structure and function . 2 PROBLEM FORMULATION . In this work we study the following network inverse problem involving undirected and weighted graphs G ( V , E ) , where V = { 1 , . . . , N } is the set of nodes ( henceforth common to all graphs ) , and E ⊆ V × V collects the edges . We get to observe a graph with symmetric adjacency matrix AO ∈ RN×N+ , that is related to a latent sparse , graphAL ∈ RN×N+ of interest via the model AO = α0I + α1AL + α2A 2 L + . . . = ∞∑ i=0 αiA i L , ( 1 ) where I denotes the N × N identity matrix . The analytic graph mapping in ( 1 ) is a polynomial in AL of possibly infinite degree , yet the Cayley-Hamilton theorem asserts it can always be equivalently reparameterized as a polynomial of degree smaller than N . Said matrix polynomials H ( A ; h ) : = ∑K k=0 hkA k , K ≤ N − 1 , with coefficients h : = [ h0 , . . . , hK ] > ∈ RK+1 , are known as shift-invariant graph convolutional filters ; see e.g. , ( Ortega et al. , 2018 ; Gama et al. , 2020b ) . Going back to the model ( 1 ) , we postulate that AO = H ( AL , h ) for some filter order K and its associated coefficients h , such that we can think of the observed network as generated via a graph convolutional process acting onAL . More pragmaticallyAO may correspond to a noisy observation ofH ( AL , h ) , and this will be clear from the context when e.g. , we estimateAO from data . Recovery of the latent graph AL is a challenging endeavour since we do not know H ( AL ; h ) , namely the parameters K or h ; see Appendix A.1 for issues of model identifiability . Suppose that AL is a realization drawn from some distribution of sparse graphs , say , for e.g. , random geometric graphs or structural brain networks from a homogeneous dataset . Then given independent training samples T : = { A ( i ) O , A ( i ) L } Ti=1 adhering to ( 1 ) , our goal is to learn a judicious parametric mapping that predicts the graph adjacency matrix ÂL = Φ ( AO ; Θ ) by minimizing a loss function L ( Θ ) : = 1 T ∑ i∈T ` ( A ( i ) L , Φ ( A ( i ) O ; Θ ) ) . ( 2 ) The loss ` is chosen to accommodate the task at hand – hinge loss for link prediction or meansquared/absolute error for the more challenging edge-weight regression problem ; see Appendix A.4 . 3 MOTIVATING APPLICATION DOMAINS . Here we outline several graph inference tasks that can be cast as the network inverse problem ( 1 ) . Latent graph structure identification from diffused signals . Our initial focus here is on identifying graphs that explain the structure of a class of network diffusion processes . Formally , let x ∈ RN be a graph signal ( i.e. , a vector of nodal features ) supported on a latent graph G with adjacencyAL . Further , letw be a zero-mean white signal with covariance matrix Σw = E [ ww > ] = I . We say that AL represents the structure of the signal x if there exists a linear network diffusion process in G that generates the signalx fromw , namelyx = ∑∞ i=0 αiA i Lw = H ( AL , h ) w. This is a fairly common generative model for random network processes ( Barrat et al. , 2008 ; DeGroot , 1974 ) . We think of the edges of G as direct ( one-hop ) relations between the elements of the signal x . The diffusion described byH ( AL , h ) generates indirect relations . In this context , the latent graph learning problem is to recover a sparse AL from a set X : = { xi } Pi=1 of P samples of x ( Segarra et al. , 2017 ) . Interestingly , from the model for x it follows that the signal covariance matrix Σx = E [ xx > ] = H2 is also a polynomial in AL ( we used Σw = I , and wrote H ←H ( AL , h ) for notational simplicity ) . The connection with ( 1 ) should now be apparent with the identificationAO = Σx . In practice , given the signals in X one would estimate the covariance matrix , e.g . via the sample covariance Σ̂x , and then aim to recover the graph AL by tackling the aforementioned network inverse problem . In this paper , we propose a fresh learning-based solution using training examples T : = { Σ̂ ( i ) x , A ( i ) L } Ti=1 . Network deconvolution and denoising . The network deconvolution problem is to identify a sparse adjacency matrix AL that encodes direct dependencies , when given an adjacency matrix AO of indirect relationships . The problem broadens the scope of e.g. , signal deconvolution to networks and can be tackled by attempting to invert the mapping AO = AL ( I − AL ) −1 = ∑∞ i=1A i L. This solution proposed in ( Feizi et al. , 2013 ) assumes a polynomial relationship as in ( 1 ) , but for the particular case of a single-pole , single-zero graph filter with very specific filter coefficients [ cf . ( 1 ) with α0 = 0 and αi = 1 , i ≥ 1 ] . This way , the indirect dependencies observed in AO arise due to the higher-order convolutive mixture terms A2L +A 3 L + . . . superimposed to the direct interactions in AL we wish to recover . Our idea in this paper is to adopt a more general , data-driven learning approach in assuming thatAO can be written as a polynomial inAL , but being agnostic to the form of the filter . Unlike the problem outlined before , here AO need not be a covariance matrix . Indeed , AO could be a corrupted graph we wish to denoise , obtained via an upstream graph learning method . Potential application domains for which supervised data T : = { A ( i ) O , A ( i ) L } Ti=1 is available include bioinformatics [ infer protein contact structure from mutual information graphs of the covariation of amino acid residues ( Feizi et al. , 2013 ) ] , social and information networks [ e.g. , learn to sparsify graphs ( Spielman & Srivastava , 2011 ) to unveil the most relevant collaborations in a social network encoding co-authorship information ( Segarra et al. , 2017 ) ] , and epidemiology ( such as contact tracing by deconvolving the graphs that model observed disease spread in a population ) . In Section 5.2 we experiment with social networks and the network neuroscience problem described next . Inferring structural brain networks from functional MRI ( fMRI ) signals . Brain connectomes encompass networks of brain regions connected by ( statistical ) functional associations ( FC ) or by anatomical white matter fiber pathways ( SC ) . The latter can be extracted from time-consuming tractography algorithms applied to diffusion MRI ( dMRI ) , which are particularly fraught due to quality issues in the data ( Yeh et al. , 2021 ) . FC represents pairwise correlation structure between blood- oxygen-level-dependent ( BOLD ) signals measured by fMRI . Deciphering the relationship between SC and FC is a very active area of research ( Abdelnour et al. , 2014 ; Honey et al. , 2009 ) and also relevant in studying neurological disorders , since it is known to vary with respect to healthy subjects in pathological contexts ( Gu et al. , 2021 ) . Traditional approaches go all the way from correlation studies ( Greicius et al. , 2008 ) to large-scale simulations of nonlinear cortical activity models ( Honey et al. , 2009 ) . More aligned with the problem addressed here , recent studies have shown that linear diffusion dynamics can reasonably model the FC-SC coupling ( Abdelnour et al. , 2014 ; Surampudi et al. , 2018 ) . Using our notation , the findings in ( Abdelnour et al. , 2014 ) suggest that the covariance AO = Σx of the functional signals ( i.e. , the FC ) is related to the the sparse SC graph AL via the model in ( 1 ) . Similarly , Liang & Wang ( 2017 ) contend the estimated FC can be represented as a weighted sum of the powers of the SC matrix , consisting of both direct and indirect effects along varying paths . There is evidence that FC links tend to exist where there is no or little structural connection ( Damoiseaux & Greicius , 2009 ) , a characteristic naturally captured by ( 1 ) . These considerations motivate adopting the proposed graph learning method to infer SC patterns from fMRI signals ( Section 5.2 ) , a significant problem for several reasons . The ability to collect only FC and get informative estimates of SC open the door to large scale studies , previously constrained by the logistical , cost , and computational resources needed to acquire both modalities .

Summary: This paper proposes an approach to estimate a latent graph $A_L$ given an observed graph $A_O$ (e.g., the covariance matrix of signals generated through a graph diffusion process). The authors posit a polynomial model and unroll proximal gradient iterations to estimate a variant of the model. The authors conduct experiments on synthetic datasets with supervised $(A_O, A_L)$ pairs as well as a neuroimaging dataset, where $A_O$ denotes the functional connectivity graph and $A_L$ denotes the structural connectivity graph. The proposed approach performs better than several baseline approaches.