GLAD: Learning Sparse Graph Recovery

1 INTRODUCTION . Recovering sparse conditional independence graphs from data is a fundamental problem in high dimensional statistics and time series analysis , and it has found applications in diverse areas . In computational biology , a sparse graph structure between gene expression data may be used to understand gene regulatory networks ; in finance , a sparse graph structure between financial timeseries may be used to understand the relationship between different financial assets . A popular formulation of the problem is an ` 1 regularization log-determinant estimation of the precision matrix . Based on this convex formulation , many algorithms have been designed to solve this problem efficiently , and one can formally prove that under a list of conditions , the solution of the optimization problem is guaranteed to recover the graph structure with high probability . However , convex optimization based approaches have their own limitations . The hyperparameters , such as the regularization parameters and learning rate , may depend on unknown constants , and need to be tuned carefully to achieve the recovery results . Furthermore , the formulation uses a single regularization parameter for all entries in the precision matrix , which may not be optimal . It is intuitive that one may obtain better recovery results by allowing the regularization parameters to vary across the entries in the precision matrix . However , such flexibility will lead to a quadratic increase in the number of hyperparameters , but it is hard for traditional approaches to search over a large number of hyperparameters . Thus , a new paradigm may be needed for designing more effective sparse recovery algorithms . Recently , there has been a surge of interest in a new paradigm of algorithm design , where algorithms are augmented with learning modules trained directly with data , rather than prescribing every step of the algorithms . This is meaningful because very often a family of optimization problems needs to be solved again and again , similar in structures but different in data . A data-driven algorithm may be able to leverage this distribution of problem instances , and learn an algorithm which performs better than traditional convex formulation . In our case , the sparse graph recovery problem may also need to be solved again and again , where the underlying graphs are different but have similar degree distribution , the magnitude of the precision matrix entries , etc . For instance , gene regulatory networks may be rewiring depending on the time and conditions , and we want to estimate them from gene expression data . Company relations may evolve over time , and we want to estimate their graph from stock data . Thus , we will also explore data-driven algorithm design in this paper . Given a task ( e.g . an optimization problem ) , an algorithm will solve it and provide a solution . Thus we can view an algorithm as a function mapping , where the input is the task-specific information ( i.e . the sample covariance matrix in our case ) and the output is the solution ( i.e . the estimated precision matrix in our case ) . However , it is very challenging to design a data-driven algorithm for precision matrix estimation . First , the input and output of the problem may be large . A neural network parameterization of direct mapping from the input covariance matrix to the output precision matrix may require as many parameters as the square of the number of dimensions . Second , there are many structure constraints in the output . The resulting precision matrix needs to be positive definite and sparse , which is not easy to enforce by a simple deep learning architecture . Third , direct mapping may result in a model with lots of parameters , and hence may require lots of data to learn . Thus a data-driven algorithm needs to be designed carefully to achieve a better bias-variance trade-off and satisfy the output constraints . In this paper , we propose a deep learning model ‘ GLAD ’ with following attributes : • Uses an unrolled Alternating Minimization ( AM ) algorithm as an inductive bias . • The regularization and the square penalty terms are parameterized as entry-wise functions of intermediate solutions , allowing GLAD to learn to perform entry-wise regularization update . • Furthermore , this data-driven algorithm is trained with a collection of problem instances in a supervised fashion , by directly comparing the algorithm outputs to the ground truth graphs . In our experiments , we show that the AM architecture provides very good inductive bias , allowing the model to learn very effective sparse graph recovery algorithm with a small amount of training data . In all cases , the learned algorithm can recover sparse graph structures with much fewer data points from a new problem , and it also works well in recovering gene regulatory networks based on realistic gene expression data generators . Related works . Belilovsky et al . ( 2017 ) considers CNN based architecture that directly maps empirical covariance matrices to estimated graph structures . Previous works have parameterized optimization algorithms as recurrent neural networks or policies in reinforcement learning . For instance , Andrychowicz et al . ( 2016 ) considered directly parameterizing optimization algorithm as an RNN based framework for learning to learn . Li & Malik ( 2016 ) approach the problem of automating algorithm design from reinforcement learning perspective and represent any particular optimization algorithm as a policy . Khalil et al . ( 2017 ) learn combinatorial optimzation over graph via deep Q-learning . These works did not consider the structures of our sparse graph recovery problem . Another interesting line of approach is to develop deep neural networks based on unfolding an iterative algorithm Gregor & LeCun ( 2010 ) ; Chen et al . ( 2018 ) ; Liu et al . ( 2018 ) . Liu et al . ( 2018 ) developed ALISTA which is based on unrolling the Iterative Shrinkage Thresholding Algorithm ( ISTA ) . Sun et al . ( 2016 ) developed ‘ ADMM-Net ’ , which is also developed for compressive sensing of MRI data . Though these seminal works were primarily developed for compressive sensing applications , they alluded to the general theme of using unrolled algorithms as inductive biases . We thus identify a suitable unrolled algorithm and leverage its inductive bias to solve the sparse graph recovery problem . 2 SPARSE GRAPH RECOVERY PROBLEM AND CONVEX FORMULATION . Givenm observations of a d-dimensional multivariate Gaussian random variableX = [ X1 , . . . , Xd ] > , the sparse graph recovery problem aims to estimate its covariance matrix Σ∗ and precision matrix Θ∗ = ( Σ∗ ) −1 . The ij-th component of Θ∗ is zero if and only if Xi and Xj are conditionally independent given the other variables { Xk } k 6=i , j . Therefore , it is popular to impose an ` 1 regularization for the estimation of Θ∗ to increase its sparsity and lead to easily interpretable models . Following Banerjee et al . ( 2008 ) , the problem is formulated as the ` 1-regularized maximum likelihood estimation Θ̂ = arg minΘ∈Sd++ − log ( det Θ ) + tr ( Σ̂Θ ) + ρ ‖Θ‖1 , off , ( 1 ) where Σ̂ is the empirical covariance matrix based on m samples , Sd++ is the space of d× d symmetric positive definite matrices ( SPD ) , and ‖Θ‖1 , off = ∑ i 6=j |Θij | is the off-diagonal ` 1 regularizer with regularization parameter ρ . This estimator is sensible even for non-GaussianX , since it is minimizing an ` 1-penalized log-determinant Bregman divergence Ravikumar et al . ( 2011 ) . The sparse precision matrix estimation problem in Eq . ( 1 ) is a convex optimization problem which can be solved by many algorithms . We give a few canonical and advanced examples which are compared in our experiments : G-ISTA . G-ISTA is a proximal gradient method , and it updates the precision matrix iteratively Θk+1 ← ηξkρ ( Θk − ξk ( Σ̂−Θ −1 k ) ) , where [ ηρ ( X ) ] ij : = sign ( Xij ) ( |Xij | − ρ ) + . ( 2 ) The step sizes ξk is determined by line search such that Θk+1 is SPD matrix Rolfs et al . ( 2012 ) . ADMM . Alternating direction methods of multiplier ( Boyd et al. , 2011 ) transform the problem into an equivalent constrained form , decouple the log-determinant term and the ` 1 regularization term , and result in the following augmented Lagrangian form with a penalty parameter λ : − log ( det Θ ) + tr ( Σ̂Θ ) + ρ ‖Z‖1 + 〈β , Θ− Z〉+ 1 2λ‖Z −Θ‖ 2 F . ( 3 ) Taking U : = β/λ as the scaled dual variable , the update rules for the ADMM algorithm are Θk+1 ← ( − Y + √ Y > Y + ( 4/λ ) I ) /2 , where Y = Σ̂/λ− Zk + Uk ( 4 ) Zk+1 ← ηρ/λ ( Θk+1 + Uk ) , Uk+1 ← Uk + Θk+1 − Zk+1 ( 5 ) BCD . Block-coordinate decent methods Friedman et al . ( 2008 ) updates each column ( and the corresponding row ) of the precision matrix iteratively by solving a sequence of lasso problems . The algorithm is very efficient for large scale problems involving thousands of variables . Apart from various algorithms , rigorous statistical analysis has also been provided for the optimal solution of the convex formulation in Eq . ( 1 ) . Ravikumar et al . ( 2011 ) established consistency of the estimator Θ̂ in Eq . ( 1 ) in terms of both Frobenius and spectral norms , at rate scaling roughly as ‖Θ̂ − Θ∗‖ = O ( ( ( d+ s ) log d/m ) 1/2 ) with high probability , where s is the number of nonzero entries in Θ∗ . This statistical analysis also reveal certain limitations of the convex formulation : The established consistency is based on a set of carefully chosen conditions , including the lower bound of sample size , the sparsity level of Θ∗ , the degree of the graph , the magnitude of the entries in the covariance matrix , and the strength of interaction between edge and non-edge in the precision matrix ( or mutual incoherence on the Hessian Γ∗ : = Σ∗ ⊗ Σ∗ ) . In practice , it may be hard to a problem to satisfy these recovery conditions . Therefore , it seems that there is still room for improving the above convex optimization algorithms for recovering the true graph structure . Prior to the data-driven paradigm for sparse recovery , since the target parameter Θ∗ is unknown , the best precision matrix recovery method is to resort to a surrogate objective function ( for instance , equation 1 ) . Optimally tuning the unknown parameter ρ is a very challenging problem in practice . Instead , we can leverage the large amount of simulation or real data and design a learning algorithm that directly optimizes the loss in equation 9 . Furthermore , since the log-determinant estimator in Eq . ( 1 ) is NOT directly optimizing the recovery objective ‖Θ̂−Θ∗‖2F , there is also a mismatch in the optimization objective and the final evaluation objective ( refer to the first experiment in section 5.1 ) . This increase the hope one may improve the results by directly optimizing the recovery objective with the algorithms learned from data . 3 LEARNING DATA-DRIVEN ALGORITHM FOR GRAPH RECOVERY . In the remainder of the paper , we will present a data-driven method to learn an algorithm for precision matrix estimation , and we call the resulting algorithm GLAD ( stands for Graph recovery Learning Algorithm using Data-driven training ) . We ask the question of Given a family of precision matrices , is it possible to improve recovery results for sparse graphs by learning a data-driven algorithm ? More formally , suppose we are given n precision matrices { Θ∗ ( i ) } ni=1 from a family G of graphs and m samples { x ( i , j ) } mj=1 associated with each Θ∗ ( i ) . These samples can be used to form n sample covariance matrices { Σ̂ ( i ) } ni=1 . We are interested in learning an algorithm for precision matrix estimation by solving a supervised learning problem , minf 1n ∑n i=1L ( GLADf ( Σ̂ ( i ) ) , Θ∗ ( i ) ) , where f is a set of parameters in GLAD ( · ) and the output of GLADf ( Σ̂ ( i ) ) is expected to be a good estimation of Θ∗ ( i ) in terms of an interested evaluation metric L. The benefit is that it can directly optimize the final evaluation metric which is related to the desired structure or graph properties of a family of problems . However , it is a challenging task to design a good parameterization of GLADf for this graph recovery problem . We will explain the challenges below and then present our solution . 3.1 CHALLENGES IN DESIGNING LEARNING MODELS In the literature on learning data-driven algorithms , most models are designed using traditional deep learning architectures , such as fully connected DNN or recurrent neural networks . But , for graph recovery problems , directly using these architectures does not work well due to the following reasons . First , using a fully connected neural network is not practical . Since both the input and the output of graph recovery problems are matrices , the number of parameters scales at least quadratically in d. Such a large number of parameters will need many input-output training pairs to provide a decent estimation . Thus some structures need to be imposed in the network to reduce the size of parameters and sample complexity . Second , structured models such as convolution neural networks ( CNNs ) have been applied to learn a mapping from Σ̂ to Θ∗ ( Belilovsky et al. , 2017 ) . Due to the structure of CNNs , the number of parameters can be much smaller than fully connected networks . However , a recovered graph should be permutation invariant with respect to the matrix rows/columns , and this constraint is very hard to be learned by CNNs , unless there are lots of samples . Also , the structure of CNN is a bias imposed on the model , and there is no guarantee why this structure may work . Third , the intermediate results produced by both fully connected networks and CNNs are not interpretable , making it hard to diagnose the learned procedures and progressively output increasingly improved precision matrix estimators . Fourth , the SPD constraint is hard to impose in traditional deep learning architectures . Although , the above limitations do suggest a list of desiderata when designing learning models : Small model size ; Minimalist learning ; Interpretable architecture ; Progressive improvement ; and SPD output . These desiderata will motivate the design of our deep architecture using unrolled algorithms . 3.2 GLAD : DEEP LEARNING MODEL BASED ON UNROLLED ALGORITHM To take into account the above desiderata , we will use an unrolled algorithm as the template for the architecture design of GLAD . The unrolled algorithm already incorporates some problem structures , such as permutation invariance and interpretable intermediate results ; but this unrolled algorithm does not traditionally have a learning component , and is typically not directly suitable for gradient-based approaches . We will leverage this inductive bias in our architecture design and augment the unrolled algorithm with suitable and flexible learning components , and then train these embedded models with stochastic gradient descent . GLAD model is based on a reformulation of the original optimization problem in Eq . ( 1 ) with a squared penalty term , and an alternating minimization ( AM ) algorithm for it . More specifically , we consider a modified optimization with a quadratic penalty parameter λ : Θ̂λ , Ẑλ : = arg minΘ , Z∈Sd++ − log ( det Θ ) + tr ( Σ̂Θ ) + ρ ‖Z‖1 + 1 2λ ‖Z −Θ‖ 2 F ( 6 ) and the alternating minimization ( AM ) method for solving it : ΘAMk+1 ← 12 ( − Y + √ Y > Y + 4λI ) , where Y = 1λ Σ̂− Z AM k ; ( 7 ) ZAMk+1 ← ηρ/λ ( ΘAMk+1 ) , ( 8 ) where ηρ/λ ( θ ) : = sign ( θ ) max ( |θ| − ρ/λ , 0 ) . The derivation of these steps are given in Appendix A . We replace the penalty constants ( ρ , λ ) by problem dependent neural networks , ρnn and Λnn . These neural networks are minimalist in terms of the number of parameters as the input dimensions are mere { 3 , 2 } for { ρnn , Λnn } and outputs a single value . Algorithm 1 summarizes the update equations for our unrolled AM based model , GLAD . Except for the parameters in ρnn and Λnn , the constant t for initialization is also a learnable scalar parameter . This unrolled algorithm with neural network augmentation can be viewed as a highly structured recurrent architecture as illustrated in Figure 1 . There are many traditional algorithms for solving graph recovery problems . We choose AM as our basis because : First , empirically , we tried models built upon other algorithms including G-ISTA , ADMM , etc , but AM-based model gives consistently better performances . Appendix C.10 & C.11 discusses different parameterizations tried . Second , and more importantly , the AM-based architecture has a nice property of maintaining Θk+1 as a SPD matrix throughout the iterations as long as λk < ∞ . Third , as we prove later in Section 4 , the AM algorithm has linear convergence rate , allowing us to use a fixed small number of iterations and still achieve small error margins . 3.3 TRAINING ALGORITHM Algorithm 1 : GLAD Function GLADcell ( Σ̂ , Θ , Z , λ ) : λ← Λnn ( ‖Z −Θ‖2F , λ ) Y ← λ−1Σ̂− Z Θ← 12 ( − Y + √ Y > Y + 4λI ) For all i , j do ρij = ρnn ( Θij , Σ̂ij , Zij ) Zij ← ηρij ( Θij ) return Θ , Z , λ Function GLAD ( Σ̂ ) : Θ0 ← ( Σ̂ + tI ) −1 , λ0 ← 1 For k = 0 to K − 1 do Θk+1 , Zk+1 , λk+1 ←GLADcell ( Σ̂ , Θk , Zk , λk ) return ΘK , ZK To learn the parameters in GLAD architecture , we will directly optimize the recovery objective function rather than using log-determinant objective . A nice property of our deep learning architecture is that each iteration of our model will output a valid precision matrix estimation . This allows us to add auxiliary losses to regularize the intermediate results of our GLAD architecture , guiding it to learn parameters which can generate a smooth solution trajectory . Specifically , we will use Frobenius norm in our experiments , and design an objective which has some resemblance to the discounted cumulative reward in reinforcement learning : min f lossf : = 1 n n∑ i=1 K∑ k=1 γK−k ∥∥∥Θ ( i ) k −Θ∗∥∥∥2 F , ( 9 ) where ( Θ ( i ) k , Z ( i ) k , λ ( i ) k ) = GLADcellf ( Σ̂ ( i ) , Θ ( i ) k−1 , Z ( i ) k−1 , λ ( i ) k−1 ) is the output of the recurrent unit GLADcell at k-th iteration , K is number of unrolled iterations , and γ ≤ 1 is a discounting factor . We will use stochastic gradient descent algorithm to train the parameters f in the GLADcell . A key step in the gradient computation is to propagate gradient through the matrix square root in the GLADcell . To do this efficiently , we make use of the property of SPD matrix that X = X1/2X1/2 , and the product rule of derivatives to obtain dX = d ( X1/2 ) X1/2 +X1/2d ( X1/2 ) . ( 10 ) The above equation is a Sylvester ’ s equation for d ( X1/2 ) . Since the derivative dX for X is easy to obtain , then the derivative of d ( X1/2 ) can be obtained by solving the Sylvester ’ s equation in ( 10 ) . The objective function in equation 9 should be understood in a similar way as in Gregor & LeCun ( 2010 ) ; Belilovsky et al . ( 2017 ) ; Liu et al . ( 2018 ) where deep architectures are designed to directly produce the sparse outputs . For GLAD architecture , a collection of input covariance matrix and ground truth sparse precision matrix pairs are available during training , either coming from simulated or real data . Thus the objective function in equation 9 is formed to directly compare the output of GLAD with the ground truth precision matrix . The goal is to train the deep architecture which can perform well for a family/distribution of input covariance matrix and ground truth sparse precision matrix pairs . The average in the objective function is over different input covariance and precision matrix pairs such that the learned architecture is able to perform well over a family of problem instances . Furthermore , each layer of our deep architecture outputs an intermediate prediction of the sparse precision matrix . The objective function takes into account all these intermediate outputs , weights the loss according to the layer of the deep architecture , and tries to progressively bring these intermediate layer outputs closer and closer to the target ground truth . 3.4 A NOTE ON GLAD ARCHITECTURE ’ S EXPRESSIVE ABILITY We note that the designed architecture , is more flexible than just learning the regularization parameters . The component in GLAD architecture corresponding to the regularization parameters are entry-wise and also adaptive to the input covariance matrix and the intermediate outputs . GLAD architecture can adaptively choose a matrix of regularization parameters . This task will be very challenging if the matrix of regularization parameters are tuned manually using cross-validation . A recent theoretical work Sun et al . ( 2018 ) also validates the choice of GLAD ’ s design .

Summary: This paper proposes an approach to data driven edge recovery for sparse gaussian mrfs. The authors propose an AM procedure for solving the l1 regularized maximum likelihood which can be unrolled and parameterized. This method is shown to converge faster at inference time than other methods and it is also far more effective in terms of training time compared to an existing data driven method. The authors provide a theoretical analysis which explains how the AM procedure should succeed and some insights on how it can potentially converge better using an adaptive model (motivating the learning part).