Learning Graph Structure from Convolutional Mixtures

1 INTRODUCTION . Inferring graphs from data to uncover latent complex information structure is a timely challenge for geometric deep learning ( Bronstein et al. , 2017 ) and graph signal processing ( Ortega et al. , 2018 ) . But it is also an opportunity , since network topology inference advances ( Dong et al. , 2019 ; Mateos et al. , 2019 ) could facilitate adoption of graph neural networks ( GNNs ) even when no input graph is available ( Hamilton , 2020 ) . The problem is also relevant when a given graph is too noisy or perturbed beyond what stable ( possibly pre-trained ) GNN architectures can effectively handle ( Gama et al. , 2020a ) . Early empirical evidence suggests that even when a graph is available , the structure could be further optimized for a downstream task ( Kazi et al. , 2020 ; Feizi et al. , 2013 ) , or else sparsified to boost computational efficiency and model interpretability ( Spielman & Srivastava , 2011 ) . In this paper , we posit a convolutional model relating observed and latent undirected graphs and formulate the graph learning task as a supervised network inverse problem ; see Section 2 for a formal problem statement . This fairly general model is motivated by various practical domains outlined in Section 3 , such as identifying the structure of network diffusion processes ( Segarra et al. , 2017 ; Pasdeloup et al. , 2018 ) , as well as network deconvolution and denoising ( Feizi et al. , 2013 ) . We propose a parameterized neural network model , termed graph deconvolution network ( GDN ) , which we train in a supervised fashion to learn the distribution of latent graphs . The architecture is derived from the principle of algorithm unrolling used to learn fast approximate solutions to inverse problems ( Gregor & LeCun , 2010 ; Sprechmann et al. , 2015 ; Monga et al. , 2021 ) , an idea that is yet to be explored in the context of graph structure identification . Since layers directly operate on , combine , and refine graph objects ( instead of nodal features ) , GDNs are inherently inductive and can generalize to graphs of different size . This allows the transfer of learning on small graphs to unseen larger graphs , which has significant implications in domains like social networks and molecular biology ( Yehudai et al. , 2021 ) . Our experiments demonstrate that GDNs are versatile to accommodate link prediction or edge-weight regression aspects of learning the graph structure , and achieve superior performance over various competing alternatives . Building on recent models of functional activity in the brain as a diffusion process over the underlying anatomical pathways ( Abdelnour et al. , 2014 ; Liang & Wang , 2017 ) , we show the applicability of GDNs to infer brain structural connectivity from functional networks obtained from the Human Connectome Project-Young Adult ( HCP-YA ) dataset . We also use GDNs to predict Facebook ties from user co-location data , outperforming relevant baselines . Related work . Network topology inference has a long history in statistics ( Dempster , 1972 ) , with noteworthy contributions for probabilistic graphical model selection ; see e.g . ( Kolaczyk , 2009 ; Friedman et al. , 2008 ; Drton & Maathuis , 2017 ) . Recent advances were propelled by graph signal processing insights through the lens of signal representation ( Dong et al. , 2019 ; Mateos et al. , 2019 ) , exploiting models of network diffusion ( Daneshmand et al. , 2014 ) , or else leveraging cardinal properties of network data such as smoothness ( Dong et al. , 2016 ; Kalofolias , 2016 ) and graph stationarity ( Segarra et al. , 2017 ; Pasdeloup et al. , 2018 ) . These works formulate ( convex ) optimization problems one has to solve for different graphs , and can lack robustness to signal model misspecifications . Scalability is an issue for the spectral-based network deconvolution approaches in ( Segarra et al. , 2017 ; Feizi et al. , 2013 ) , that require computationally-expensive eigendecompositions of the input graph for each problem instance . Moreover , none of these methods advocate a supervised learning paradigm to learn distributions over adjacency matrices . When it comes to this latter objective , deep generative models ( Liao et al. , 2019 ; Wang et al. , 2018 ; Li et al. , 2018 ) are typically trained in an unsupervised fashion , with the different goal of sampling from the learnt distribution . Most of these approaches learn node embeddings and are inherently transductive . Recently , so-termed latent graph learning has been shown effective in obtaining better task-driven representations of relational data for machine learning ( ML ) applications ( Wang et al. , 2019 ; Kazi et al. , 2020 ; Veličković et al. , 2020 ) , or to learn interactions among coupled dynamical systems ( Kipf et al. , 2018 ) . Summary of contributions . We introduce GDNs , a supervised learning model capable of recovering latent graph structure from observations of its convolutional mixtures , i.e. , related graphs containing spurious , indirect relationships . Our experiments on synthetic and real datasets demonstrate the effectiveness of GDNs for said task . They also showcase the model ’ s versatility to incorporate domain-specific topology information about the sought graphs . On synthetic data , GDNs outperform comparable methods on link prediction and edge-weight regression tasks across different randomgraph ensembles , while incurring a markedly lower ( post-training ) computational cost and inference time . GDNs are inductive and learnt models transfer across graphs of different sizes . We verify they exhibit minimal performance degradation even when tested on graphs 60× larger . Finally , using GDNs we propose a novel ML pipeline to learn whole brain structural connectivity ( SC ) from functional connectivity ( FC ) , a challenging and timely problem in network neuroscience . Results on the high-quality HCP-YA imaging dataset show that GDN performs well on specific brain subnetworks that are known to be relatively less correlated with the corresponding FC due to ageing-related effects – a testament to the model ’ s robustness and expressive power . Overall , results here support the promising prospect of using graph representation learning to integrate brain structure and function . 2 PROBLEM FORMULATION . In this work we study the following network inverse problem involving undirected and weighted graphs G ( V , E ) , where V = { 1 , . . . , N } is the set of nodes ( henceforth common to all graphs ) , and E ⊆ V × V collects the edges . We get to observe a graph with symmetric adjacency matrix AO ∈ RN×N+ , that is related to a latent sparse , graphAL ∈ RN×N+ of interest via the model AO = α0I + α1AL + α2A 2 L + . . . = ∞∑ i=0 αiA i L , ( 1 ) where I denotes the N × N identity matrix . The analytic graph mapping in ( 1 ) is a polynomial in AL of possibly infinite degree , yet the Cayley-Hamilton theorem asserts it can always be equivalently reparameterized as a polynomial of degree smaller than N . Said matrix polynomials H ( A ; h ) : = ∑K k=0 hkA k , K ≤ N − 1 , with coefficients h : = [ h0 , . . . , hK ] > ∈ RK+1 , are known as shift-invariant graph convolutional filters ; see e.g. , ( Ortega et al. , 2018 ; Gama et al. , 2020b ) . Going back to the model ( 1 ) , we postulate that AO = H ( AL , h ) for some filter order K and its associated coefficients h , such that we can think of the observed network as generated via a graph convolutional process acting onAL . More pragmaticallyAO may correspond to a noisy observation ofH ( AL , h ) , and this will be clear from the context when e.g. , we estimateAO from data . Recovery of the latent graph AL is a challenging endeavour since we do not know H ( AL ; h ) , namely the parameters K or h ; see Appendix A.1 for issues of model identifiability . Suppose that AL is a realization drawn from some distribution of sparse graphs , say , for e.g. , random geometric graphs or structural brain networks from a homogeneous dataset . Then given independent training samples T : = { A ( i ) O , A ( i ) L } Ti=1 adhering to ( 1 ) , our goal is to learn a judicious parametric mapping that predicts the graph adjacency matrix ÂL = Φ ( AO ; Θ ) by minimizing a loss function L ( Θ ) : = 1 T ∑ i∈T ` ( A ( i ) L , Φ ( A ( i ) O ; Θ ) ) . ( 2 ) The loss ` is chosen to accommodate the task at hand – hinge loss for link prediction or meansquared/absolute error for the more challenging edge-weight regression problem ; see Appendix A.4 . 3 MOTIVATING APPLICATION DOMAINS . Here we outline several graph inference tasks that can be cast as the network inverse problem ( 1 ) . Latent graph structure identification from diffused signals . Our initial focus here is on identifying graphs that explain the structure of a class of network diffusion processes . Formally , let x ∈ RN be a graph signal ( i.e. , a vector of nodal features ) supported on a latent graph G with adjacencyAL . Further , letw be a zero-mean white signal with covariance matrix Σw = E [ ww > ] = I . We say that AL represents the structure of the signal x if there exists a linear network diffusion process in G that generates the signalx fromw , namelyx = ∑∞ i=0 αiA i Lw = H ( AL , h ) w. This is a fairly common generative model for random network processes ( Barrat et al. , 2008 ; DeGroot , 1974 ) . We think of the edges of G as direct ( one-hop ) relations between the elements of the signal x . The diffusion described byH ( AL , h ) generates indirect relations . In this context , the latent graph learning problem is to recover a sparse AL from a set X : = { xi } Pi=1 of P samples of x ( Segarra et al. , 2017 ) . Interestingly , from the model for x it follows that the signal covariance matrix Σx = E [ xx > ] = H2 is also a polynomial in AL ( we used Σw = I , and wrote H ←H ( AL , h ) for notational simplicity ) . The connection with ( 1 ) should now be apparent with the identificationAO = Σx . In practice , given the signals in X one would estimate the covariance matrix , e.g . via the sample covariance Σ̂x , and then aim to recover the graph AL by tackling the aforementioned network inverse problem . In this paper , we propose a fresh learning-based solution using training examples T : = { Σ̂ ( i ) x , A ( i ) L } Ti=1 . Network deconvolution and denoising . The network deconvolution problem is to identify a sparse adjacency matrix AL that encodes direct dependencies , when given an adjacency matrix AO of indirect relationships . The problem broadens the scope of e.g. , signal deconvolution to networks and can be tackled by attempting to invert the mapping AO = AL ( I − AL ) −1 = ∑∞ i=1A i L. This solution proposed in ( Feizi et al. , 2013 ) assumes a polynomial relationship as in ( 1 ) , but for the particular case of a single-pole , single-zero graph filter with very specific filter coefficients [ cf . ( 1 ) with α0 = 0 and αi = 1 , i ≥ 1 ] . This way , the indirect dependencies observed in AO arise due to the higher-order convolutive mixture terms A2L +A 3 L + . . . superimposed to the direct interactions in AL we wish to recover . Our idea in this paper is to adopt a more general , data-driven learning approach in assuming thatAO can be written as a polynomial inAL , but being agnostic to the form of the filter . Unlike the problem outlined before , here AO need not be a covariance matrix . Indeed , AO could be a corrupted graph we wish to denoise , obtained via an upstream graph learning method . Potential application domains for which supervised data T : = { A ( i ) O , A ( i ) L } Ti=1 is available include bioinformatics [ infer protein contact structure from mutual information graphs of the covariation of amino acid residues ( Feizi et al. , 2013 ) ] , social and information networks [ e.g. , learn to sparsify graphs ( Spielman & Srivastava , 2011 ) to unveil the most relevant collaborations in a social network encoding co-authorship information ( Segarra et al. , 2017 ) ] , and epidemiology ( such as contact tracing by deconvolving the graphs that model observed disease spread in a population ) . In Section 5.2 we experiment with social networks and the network neuroscience problem described next . Inferring structural brain networks from functional MRI ( fMRI ) signals . Brain connectomes encompass networks of brain regions connected by ( statistical ) functional associations ( FC ) or by anatomical white matter fiber pathways ( SC ) . The latter can be extracted from time-consuming tractography algorithms applied to diffusion MRI ( dMRI ) , which are particularly fraught due to quality issues in the data ( Yeh et al. , 2021 ) . FC represents pairwise correlation structure between blood- oxygen-level-dependent ( BOLD ) signals measured by fMRI . Deciphering the relationship between SC and FC is a very active area of research ( Abdelnour et al. , 2014 ; Honey et al. , 2009 ) and also relevant in studying neurological disorders , since it is known to vary with respect to healthy subjects in pathological contexts ( Gu et al. , 2021 ) . Traditional approaches go all the way from correlation studies ( Greicius et al. , 2008 ) to large-scale simulations of nonlinear cortical activity models ( Honey et al. , 2009 ) . More aligned with the problem addressed here , recent studies have shown that linear diffusion dynamics can reasonably model the FC-SC coupling ( Abdelnour et al. , 2014 ; Surampudi et al. , 2018 ) . Using our notation , the findings in ( Abdelnour et al. , 2014 ) suggest that the covariance AO = Σx of the functional signals ( i.e. , the FC ) is related to the the sparse SC graph AL via the model in ( 1 ) . Similarly , Liang & Wang ( 2017 ) contend the estimated FC can be represented as a weighted sum of the powers of the SC matrix , consisting of both direct and indirect effects along varying paths . There is evidence that FC links tend to exist where there is no or little structural connection ( Damoiseaux & Greicius , 2009 ) , a characteristic naturally captured by ( 1 ) . These considerations motivate adopting the proposed graph learning method to infer SC patterns from fMRI signals ( Section 5.2 ) , a significant problem for several reasons . The ability to collect only FC and get informative estimates of SC open the door to large scale studies , previously constrained by the logistical , cost , and computational resources needed to acquire both modalities .

Summary: This paper proposes a novel solution to infer the graph structure. It starts with the unrolling algorithm for graph structure inference problem, and then adopt the proximal gradient iterative solutions. The expressiveness is augmented by parameterized with deep neural network, called Graph Deconvolution Network (GDN).