Principal Network Analysis

Many applications collect a large number of time series, for example, temperature continuously monitored by weather stations across the US or neural activity recorded by an array of electrical probes. These data are often referred to as unstructured. A first task in their analytics is often to derive a low dimensional representation – a graph or discrete manifold – that describes the interrelations among the time series and their intrarelations across time. In general, the underlying graphs can be directed and weighted, possibly capturing the strengths of causal relations, not just the binary existence of reciprocal correlations. Furthermore, the processes generating the data may be non-linear and observed in the presence of unmodeled phenomena or unmeasured agents in a complex networked system. Finally, the networks describing the processes may themselves vary through time. In many scenarios, there may be good reasons to believe that the graphs are only able to vary as linear combinations of a set of “principal graphs” that are fundamental to the system. We would then be able to characterize each principal network individually to make sense of the ensemble and analyze the behaviors of the interacting entities. This thesis acts as a roadmap of computationally tractable approaches for learning graphs that provide structure to data. It culminates in a framework that addresses these challenges when estimating time-varying graphs from collections of time series. Analyses are carried out to justify the various models proposed along the way and to characterize their performance. Experiments are performed on synthetic and real datasets to highlight their effectiveness and to illustrate their limitations. Acknowledgments I could not have gotten to this point without the unwavering love and support from my parents Jingjing and Renwei and my brother Vinny. Thank you for listening, passing on advice when I need it, and being there when it matters. I would not be here if not for the wisdom and guidance of my committee. Especially to my advisor Professor José Moura, thank you for providing just the right balance of gentle nudges forward, autonomy to pursue my own research interests, a nice environment in which to do so, encouragement in the face of adversity, intellectual discussions on almost any topic, sprinkled in with valuable life lessons. I am also grateful to Professor Jelena Kovačević, Professor Rob Kass, and Professor Pradeep Ravikumar for serving on my committee. Thank you for the profound insights and suggestions to help me make this thesis the best I could. I should not have made it this far without a strong group of friends and colleagues to lean on. Fortunately I did not have to. Big thanks to my groupmates past and present, Nick O’Donoghue, Augusto Santos, Joel Harley, Kyle Anderson, Subhro Das, June Zhang, Joya Deri, Stephen Kruzick, Evgeny Toropov, Lynna Gui, Shanghang Zhang, Yuan Chen, Satwik Kottur, Mark Cheung, Raj Das, John Shi, and Jian Du, for showing me the ropes and joining in with my shenanigans. Praise be to Porter B party bandits, Anit Sahu, Vishwa Saragadam, Chia-Yin Tsai, Jian Wang, John Costanzo, Yi Hua, Brian Swenson, Hadi Amini, Javad Mohammadi, Siheng Chen, Yaoqing Yang, Vince Monardo, Harlin Lee, Nikos Arechiga, and Rohan Chabuskwar, who brought light to the basement. And thanks to Devon Rosner, Alexis Hakimi, Ahmed Kirmani, Andrea Colaço, Ben Shih, Ray Carley, Clay Long, Mats Forssell, Serim Park, Stephen Siena, Sean Yen, Paola Buitrago, Hyesoo Yang, Anson Wang, Mai Sawada, Vova Katson, Stas Mamontov, and Steven and Hikari Aday, for helping keep me sane. Finally, thanks to David and LaVerne Owen-Barakat for their generous support of my studies through their graduate fellowship, which opened up many opportunities for me personally and professionally. Also, large parts of this work were made possible by NSF grants CCF 1011903 and CCF 1513936.