Learning Spatial-Temporal Varying Graphs with Applications to Climate Data Analysis

An important challenge in understanding climate change is to uncover the dependency relationships between various climate observations and forcing factors. Graphical lasso, a recently proposed `1 penalty based structure learning algorithm, has been proven successful for learning underlying dependency structures for the data drawn from a multivariate Gaussian distribution. However, climatological data often turn out to be non-Gaussian, e.g. cloud cover, precipitation, etc. In this paper, we examine nonparametric learning methods to address this challenge. In particular, we develop a methodology to learn dynamic graph structures from spatial-temporal data so that the graph structures at adjacent time or locations are similar. Experimental results demonstrate that our method not only recovers the underlying graph well but also captures the smooth variation properties on both synthetic data and climate data. Introduction Climate change poses many critical socio-technological issues in the new century (IPCC 2007). An important challenge in understanding climate change is to uncover the dependency relationships between the various climate observations and forcing factors, which can be of either natural or anthropogenic (human) origin, e.g. to assess which parameters are mostly responsible for climate change. Graph is one of the most natural representations of dependency relationships among multiple variables. There have been extensive studies on learning graph structures that are invariant over time. In particular, `1 penalty based learning algorithms, such as graphical lasso, establish themselves as one of the most promising techniques for structure learning, especially for data with inherent sparse graph structures (Meinshausen and Bühlmann 2006; Yuan and Lin 2007) and have been successfully applied in diverse areas, such as gene regulatory network discovery (Friedman 2004), social network analysis (Goldenberg and Moore 2005) and so on. Very recently, several methods have been proposed to model time-evolving graphs with applications from gene regulatory network analysis (Song, Kolar, and Xing 2009), financial data analysis (Xuan and Murphy 2007) to oil-production Copyright c © 2010, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. monitoring system (Liu, Kalagnanam, and Johnsen 2009). Most of the existing methods assume that the data are drawn from a multivariate Gaussian distribution at each time stamp and then estimate the graphs on a chain of time. Compared with the existing approaches for graph structure learning, there are two major challenges associated with climate data: one is that meteorological or climatological data often turn out to be non-Gaussian, e.g. precipitation, cloud cover, and relative humidity, which belong to bounded or skewed distributions (Boucharel et al. 2009); the other is the smooth variation property, i.e. the graph structures may vary over temporal and/or spatial scales, but the graphs at adjacent time or locations should be similar. In this paper, we present a nonparametric approach with kernel weighting techniques to address these two challenges for spatial-temporal data in climate applications. Specifically, for a fixed time t and location s, we propose to adopt a two-stage procedure: (1) instead of blindly assuming that the data follow Gaussian or any other parametric distributions, we learn a set of marginal functions which can transform the original data into a space where they are normally distributed; (2) we construct the covariance matrix for t and s via a kernel weighted combination of all the data at different time and locations. Then the state-of-the-art graph structure learning algorithm, “graphical lasso” (Yuan and Lin 2007; Friedman, Hastie, and Tibshirani 2008), can be applied to uncover the underlying graph structure. It is worthwhile noting that our kernel weighting techniques are very flexible, i.e. they can account for smooth variation in many types (e.g. altitude) besides time and space. To the best of our knowledge, it is the first practical method for learning nonstationary graph structures without assuming any parametric underlying distributions. Preliminary We concern ourselves with the problem in learning graph structures which vary in both temporal and spatial domains. At each time t and location s, we take n i.i.d. observations on p random variables which are denoted as {Xts i }i=1, where each X i := (X ts i1 , . . . X ts ip) T ∈ R is a p dimensional vector. Taking the climate data for example, we may independently measure several factors (variables), such as temperature, precipitation, carbon-dioxide (CO2), at each location spreading at different time in a year. Our goal is to explore the dependency relationships among these variables over time and locations. Markov Random Fields (MRFs) have been widely adopted for modeling dependency relationships (Kinderman and Snell 1980). For a fixed time and location, denote each observation as a p-dimensional random vector X = (X1, . . . Xp). We encode the structure of X with an undirected graph G = (V, E), where each node u in the vertex set V = {v1, . . . vp} corresponds to a component of X. The edge set encodes conditional independencies among components of X. More precisely, the edge between (u, v) is excluded from E if and only if Xu is conditionally independent of Xv given the rest of variables V\u,v ≡ {Xi, 1 ≤ i ≤ p, i 6= u, v}: (u, v) 6∈ E ⇔ Xu ⊥⊥ Xv |V\u,v (1) A large body of literature assumes that X follows a multivariate Gaussian distribution, N(μ,Σ), with the mean vector μ and the covariance matrix Σ. Let Ω = Σ−1 be the inverse of the covariance matrix (a.k.a. the precision matrix). One good property of multivariate Gaussian distributions is that Xu ⊥⊥ Xv |V\u,v if and only if Ωuv = 0 (Lauritzen 1996). Under the Gaussian assumption, we may deduce conditional independencies by estimating the inverse covariance matrix. In real world applications, many variables are conditionally independent given others. Therefore, only a few essential edges should appear in the estimated graph. In other words, the estimated inverse covariance matrix Ω̂ should be sparse with many zero elements. Inspired by the success of “lasso” for linear models, Yuan and Lin proposed “graphical lasso” to obtain a sparse Ω̂ by minimizing the negative log-likelihood with `1 penalization on Ω̂ (Yuan and Lin 2007). More precisely, let {X1,X2, . . .Xn} be n random samples from N(μ,Σ), where each Xi ∈ R and let Σ̂ be the estimated covariance matrix using maximum likelihood: