Reconstructing Hidden Regulatory Layers by Network Component Analysis : Theory and Application

The authors recently introduced a framework, named Network Component Analysis (NCA), for the reconstruction of the dynamics of transcriptional regulat ors activities from gene expression assays. In this paper, our goal is to characterize NCA as a general purpose ne twork and signal reconstruction technique: given only the noisy output signals of a multi-dimensional l inear system and certain a-priori knowledge about the connectivity among the inputs and the outputs, the method is capable of reconstructing both the input signals as well as the unknown connectivity coeffic i nts. In particular, the following aspects of NCA are investigated: I) The sufficient conditions on the a-p riori connectivity information (required for successful reconstructions via NCA) are made less stringen t, allowing easier verification of whether a network topology is identifiable, as well as extending the cl ass of identifiable systems. II) We show that the two-stage least square iterative procedure used in NCA i dentifies stationary points of the likelihood function, under gaussian noise assumption. III) A framewor k f r the simultaneous reconstruction of multiple regulatory sub-networks is introduced, thus over coming one of the limitations of the original formulation of the decomposition, occurring for small samp le size data. A set of monte carlo simulations with synthetic data suggest that the approach is indeed capa ble of accurately reconstructing regulatory signals when these are the input of large-scale networks tha t satisfy the suggested identifiability criteria, even under fairly noisy conditions. The sensitivity of the r econstructed signals to inaccuracies in the hypothesized network topology is also investigated. The re sults obtained in [1] by applying NCA to experimental gene expression measurements of the bacteriu m Escherichia coliare extended to the Riccardo Boscolo and Vwani P. Roychowdhury are with the Electrical En gineering Department, University of California Los Angeles (e-mail:{riccardo,vwani }@ee.ucla.edu). Chiara Sabatti is with the Department of Human Genetics and Statistics, Unive rsity of California Los Angeles (e-mail: csabatti@mednet.ucla.edu). James C. Liao is with the Department of Chemical Engineering, University of California Los Angeles (e-mail: liaoj@ucla.edu). IEEE TRANS. ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 2 reconstruction of multiple regulatory sub-networks with p artially overlapping sets of transcriptional regulators. Index Terms System identification, Biology and genetics, Network model s, Data analysis. I. I NTRODUCTION Recent advances in biotechnology have resulted in the introd uction of high-throughput techniques for the measurement of biological signals. An exampl e of such technologies is DNA microarray assays [2], which allow simultaneous monitorin g of the expression levels of several thousand genes in an organism. Such increase in the amount of da a made available to biologists has driven a parallel effort aimed at developing informatio n processing tools required to analyze such data sets. Accordingly, the last few years have witness ed a rapid increase in the introduction of new statistical tools and associated computational meth odologies for the analysis as well as modeling of biological systems [3]–[5]. The task of extracting information about the structure and d ynamics of intraand inter-cellular processes from these large-scale data sets has, however, pr oven to be difficult, and the reasons are quite apparent: the observed signals are the outputs of a com plex stochastic dynamical system involving a considerable number of hidden factors. The appr oaches that have been adopted so far can be broadly classified into two categories, namely, no n-parametric and parametric. In the non-parametric case, no generative model for the signal s is assumed, and inferences about intracellular mechanisms are based on different measures o f dependencies (or lack thereof) among the signals themselves. For example, works based on cluster ing of genes using their expression profiles [6] and on extracting regulatory information using Bayesian statistics [7] fall under this category. While these methods have been applied successfull y to the problem of finding patterns of co-expression between genes, it is well known that such me thods are limited in the fact that they do not provide means for including specific biological m odeling assumptions that could be derived from available a-priori knowledge on the system und er study. In a parametric approach, on the other hand, an a priori set of generative models is assumed and the parameters of these models are estimated from the dat a set. One such example is provided in [8], where the experiment design involves measurements o f the outputs of a targeted pathway in IEEE TRANS. ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 3 response to perturbations in the inputs, and the objective c oncerns the determination of causative links among the input and output signals. A linearized state -space model for the underlying mechanism is assumed and an overall sparsity constraint on t he connectivity is imposed, viz. each output signal is influenced by at most K out ofN perturbed inputs. The inference procedure is then centered around the issue of selecting a sparse conne ctivity pattern that best fits the data. In many experiments one does not have the capability of selec tively and precisely perturbing a significant number of the input signals. Instead, one has ac cess to only a subset of the system output signals measured under different conditions, which ave the overall effect of perturbing the physiological state of the cells in unidentified ways. In DNA microarray experiments, for example, one can monitor the expression level of genes, whic h are modulated by a hidden set of transcriptional regulatory mechanisms, including inte r-c llular and intra-cellular signaling, co-activation mechanisms, and competitive binding, just t o mention a few. It is unclear how one might infer characteristics of the hidden mechanisms an d signals from only the expression data. Even if one were to assume a linearized model for the int eractions among the activated transcription factors and the genes, one will be left with th e intractable problem of simultaneously estimating the parameters of the linearized system ( i.e. the input-output connectivity and the related strengths), as well as the hidden regulatory signal s. Hence, one needs to impose further constraints on the linearized model to make the inverse prob lem solvable. In [1], [9] the authors introduced a data decomposition tech nique, named Network Component Analysis (NCA), which uses a certain type of a-priori knowled g about the connectivity pattern among the input and output signals, in order to reconstruct b oth the network input signals and the strengths of its connections, when only the output si gnals are accessible. In the case of transcriptional regulation, the a-priori knowledge on tra nscription factor (TF) binding sites affinity provides information about whether a particular TF-gene re gulatory link is significant or not. For example, if the activated form of a TF is known not to bind sign ificantly to the binding sites in the promoter region of a gene, then one can set the corresponding parameter in the linearized model to zero. Data on potential TF-gene interactions (or absence ther of) is readily available for several prokaryotes as well as eukaryotes in the form of publicly ava il ble databases. Thus, the NCA decomposition technique effectively combines available s tructural data with the measurement of the outputs of the system (in this case the gene expression le vels), in order to estimate certain hidden quantities of the regulatory network, namely the act ivity levels of the major transcriptional IEEE TRANS. ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS 4 regulators, as well as the relative control strength they ex rcise on different genes. It is worthwhile to note that while well-known linear decomposition techniq ues, such as Principal Component Analysis (PCA) [10] or Independent Component Analysis [11], [ 2] have found application in the analysis of certain gene expression and other biologica l dat sets [13]–[15], such methods are not suitable for the problem of recovering hidden mechanism s, as defined in this paper. Rather than using a priori biological knowledge to make the decompo sition problem solvable, these methods impose mathematical/statistical constraints on t he input signals ( viz., orthogonality for PCA and statistical independence for ICA) and yield potential ly dense connectivity patterns, both of which are not representative of the underlying biolo gical system. In this paper, we derive certain novel results on the estimat ion problem associated with NCA and demonstrate its wide applicability as a generalized dec omposition technique. In particular, we analyze systematically for the first time the performance characteristics of the NCA decomposition as a function of the measurement error as well as of t he complexity of the regulatory network, in the ideal scenario when the model linearity is no t vi lated. The results obtained in a large-scale set of monte carlo simulations demonstrate in deed the efficacy of the method even in those cases when the modeling assumptions are subject to n oise perturbations. In Section III, we analyze the sufficient conditions for iden tifiability, i.e., when does one have enough prior knowledge, in terms of the absences of links in t he networks, so that one can perform the intended decomposition. We derive a new and redu ced set of sufficient conditions for system identifiability. We then establish a statistical fr mework for NCA and show that the iterative method introduced in [9] for estimating the re gulatory signals and the connection strengths from the observed data is equivalent to a particul ar case of Maximum Likelihood (ML) e