abstract: Inferring the coupling structure of complex systems from time series data in general by means of statistical and information-theoretic techniques is a challenging problem in applied science. The reliability of statistical inferences requires the construction of suitable information-theoretic measures that take into account both direct and indirect influences, manifest in the form of information flows, between the components within the system. In this work, we present an application of the optimal causation entropy (oCSE) principle to identify the coupling structure of a synthetic biological system, the repressilator. Specifically, when the system reaches an equilibrium state, we use a stochastic perturbation approach to extract time series data that approximate a linear stochastic process. Then, we present and jointly apply the aggregative discovery and progressive removal algorithms based on the oCSE principle to infer the coupling structure of the system from the measured data. Finally, we show that the success rate of our coupling inferences not only improves with the amount of available data, but it also increases with a higher frequency of sampling and is especially immune to false positives.
author: Jie Sun; Carlo Cafaro; Erik M. Bollt
title: Identifying Coupling Structure in Complex Systems through the Optimal Causation Entropy Principle

# Introduction

Deducing equations of dynamics from empirical observations is fundamental in science. In real-world experiments, we gather data of the state of a system. Then, to achieve the comprehension of the mechanisms behind the system dynamics, we often need to reconstruct the underlying dynamical equations from the measured data. For example, the laws of celestial mechanics were deduced based on observations of planet trajectories ; the forms of chemical equations were inferred upon empirical reaction relations and kinetics ; the principles of economics were uncovered through market data analysis . Despite such important accomplishments, the general problem of identifying dynamical equations from data is a challenging one. Early efforts, such as the one presented in , utilized embedding theory and relative entropy to reconstruct the deterministic component of a low-dimensional dynamical system.

Later, more systematic methods were developed to design optimal models from a set of basis functions, where model quality was quantified in various ways, such as the Euclidean norm of the error , the length of the prediction time window (chaotic shadowing)  or the sparsity in the model terms (compressive sensing) . Each method achieved success in a range of systems, but none is generally applicable. This is, in fact, not surprising in light of the recent work that showed that the identification of exact dynamical equations from data is NP hard and, therefore, unlikely to be solved efficiently .

Fortunately, in many applications, the problem we face is not necessarily the extraction of exact equations, but rather, uncovering the cause-and-effect relationships (*i.e.*, direct coupling structure) among the components within a complex system. For example, in medical diagnosis, the primary goal is to identify the causes of a disease and/or the roots of a disorder, so as to prescribe effective treatments. In structural health monitoring, the main objective is to locate the defects that could cause abrupt changes of the connectivity structure and adverse the performance of the system. Due to its wide range of applications, the problem of inferring causal relationships from observational data has attracted broad attention over the past few decades .

Among the various notions of causality, we adopt the one originally proposed by Granger, which relies on two basic principles :

1.  The cause should occur before the effect (caused);

2.  The causal process should carry information (unavailable in other processes) about the effect.

A causal relationship needs to satisfy both requirements. See Figure  as a schematic illustration.

The classical Granger causality is limited to coupled linear systems, while most recently developed methods based on information-theoretic measures are applicable to virtually any model, although their effectiveness relies on the abundance of data. Notably, transfer entropy (TE), a particular type of conditional mutual information, was introduced to quantify the (asymmetric) information flow between pairs of components within a system  and further developed to detect the directionality of coupling in systems with two components . The application of these pairwise inference methods to identify direct coupling in systems with more than two components warrants caution: the inferred couplings are often incorrect regardless of the amount of available data . In fact, Granger's second principle implies that a true causal relationship should remain effective upon the removal of all other possible causes. As a consequence, an inference method cannot correctly identify the direct coupling between two components in a system without appropriate conditioning on the remaining components .

Inferring causal relationships in large-scale complex systems is challenging. For a candidate causal relationship, one needs to effectively determine whether the cause and effect is real or is due to the presence of other variables in the system. A common approach is to test the relative independence between the potential cause and effect conditioned on all other variables, as demonstrated in linear models . Although analytically rigorous, this approach requires the estimation of the related inference measures for high dimensional random variables from (often limited available) data, and therefore, suffers the curse of dimensionality . Many heuristic alternatives have been proposed to address this issue. The essence of these alternative approaches is to repeatedly measure the relative independence between the cause and effect conditioned on a combination of the other variables, often starting with subsets of only a few variables . As soon as one such combination renders the proposed cause and effect independent, the proposed relationship is rejected, and there is no need to condition on subsets with more variables. The advantage of such an approach is that it reduces the dimensionality of the sample space if the proposed relationship is rejected at some point. However, if the relationship is not rejected, then the algorithm will continue, potentially having to enumerate over all subsets of the other variables. In this scenario, regardless of the dimensionality of the sample space, the combinatorial search itself is often computationally infeasible for large or even moderate-size systems.

In our recent work in , we introduced the concept of causation entropy (CSE), proposed and proved the optimal causation entropy (oCSE) principle and presented efficient algorithms to infer causal relationships within large complex systems. In particular, we showed in that by a combination of the aggregate discovery and progressive removal of variables, all causal relationships can be correctly inferred in an computational feasible and data-efficient manner. We repeat for emphasis that without properly accounting for multiple interactions and conditioning accordingly, erroneous or irrelevant casual influences may be inferred, and specifically, any pairwise-only method will inherent the problem that many false-positive connections will arise. The design of oCSE specifically addresses this issue.

In this paper, we focus on the problem of inferring the coupling structure in synthetic biological systems. When the system reaches an equilibrium state, we employ random perturbations to extract time series that approximate a linear Gaussian stochastic process. Then, we apply the oCSE principle to infer the system's coupling structure from the measured data. Finally, we show that the success rate of causal inferences not only improves with the amount of available data, but it also increases with the higher frequency of sampling.

# Inference of Direct Coupling Structure through Optimal Causation Entropy

To infer causal structures in complex systems, we clearly need to specify the mathematical assumptions under which the task is to be accomplished. Accurate mathematical modeling of complex systems demands taking into account the coupling of neglected degrees of freedom or, more generally, the fluctuations of external fields that describe the environment interacting with the system itself . This requirement can be addressed in a phenomenological manner by adding noise in deterministic dynamical models. The addition of noise, in turn, generally leads to the stochastic process formalism used in the modeling of natural phenomena . We study the system in a probabilistic framework. Suppose that the system contains $n$ components, $\mathcal{V}=\{1,2,\dots,n\}$. Each component $i$ is assumed to be influenced by a unique set of components, denoted by $N_i$. Let: $$X_t=[X^{(1)}_t,X^{(2)}_t,\dots,X^{(n)}_t]\vspace{-3pt}$$ be a random variable that describes the state of the system at time $t$. For a subset $K=\{k_1,k_2,\dots,k_q\}\subset\mathcal {V}$, we define: $$X^{(K)}_t\equiv[X^{(k_1)}_t,X^{(k_2)}_t,\dots,X^{(k_q)}_t].$$

## Markov Conditions

We assume that the system undergoes a stochastic process with the following Markov conditions . $$\label{eq:processconds}
\begin{cases}
\mbox{{(i)} {Temporally Markov}:~}\\
\quad\quad\quad\quad p(X_{t}|X_{t-1},X_{t-2},\dots) = p(X_{t}|X_{t-1})= p(X_{t'}|X_{t'-1})~\mbox{for any $t$ and $t'$.}\\
\mbox{{(ii)} {Spatially Markov}:~}\\
\quad\quad\quad\quad p(X^{(i)}_{t}|X_{t-1})=p(X^{(i)}_t|X^{(N_i)}_{t-1})~\mbox{for any $i$.}\\
\mbox{{(iii)} {Faithfully Markov}:~}\\
{
\quad\quad\quad\quad p(X^{(i)}_{t}|X^{(K)}_{t-1})\neq p(X^{(i)}_t|X^{(L)}_{t-1})~\mbox{whenever
$(K\cap N_i)\neq(L\cap N_i)$.}}
\end{cases}$$ Here, $p(\cdot|\cdot)$ denotes conditional probability. The relationship between two probability density functions $p_1$ and $p_2$ is denoted as $``p_1=p_2"$ iff they equal almost everywhere, and $``p_1\neq p_2"$ iff there is a set of positive measure on which the two functions do not equal. Note that the Markov conditions stated in Equation  ensure that for each component $i$, there is a unique set of components $N_i$ that renders the rest of the system irrelevant in making inference about $X^{(i)}$ and each individual component in $N_i$ presents an observable cause regardless of the presence or absence of the other components.

Although several complex systems can be properly modeled in terms of Markov processes , we cannot avoid recalling that, after all, non-Markov is the rule and Markov is only the exception in nature . Therefore, it is important to develop a theoretical framework suitable for identifying coupling structures in complex systems driven by correlated fluctuations where memory effects cannot be neglected. It is in fact possible to relax the assumptions in Equation  to a stochastic process with finite or vanishing memory, as discussed in the last section of the paper.

The problem of inferring direct (or causal) couplings can be stated as follows. Given time series data $\{x^{(i)}_t\}$ ($i=1,2,\dots,n;t=1,2,\dots,T$) drawn from a stochastic process that fulfills the Markov conditions in Equation , the goal is to uncover the set of causal components $N_i$ for each $i$. We solve this problem by means of algorithms that implement the oCSE principle.

## Causation Entropy

We follow Shannon's definition of entropy to quantify the uncertainty of a random variable. In particular, the entropy of a continuous random variable $X$ is defined as : $$\label{eq:hX}
    h(X) \equiv -\int p(x)\log p(x)dx,$$ where $p(x)$ is the probability density function of $X$. The conditional entropy of $X$ given $Y$ is defined as : $$\label{eq:hXY}
    h(X|Y)\equiv-\int p(x,y)\log p(x|y)dxdy.\vspace{3pt}$$ Equations  and  are valid for both univariate and multivariate random variables. Consider a stochastic process. Let $I$, $J$ and $K$ be arbitrary sets of components within the system. We propose to quantify the influence of $J$ on $I$ conditioned upon $K$ via the CSE: $$\label{eq:defcse}
    C_{J\rightarrow I|K} = \lim_{t\rightarrow\infty}[h(X^{(I)}_{t+1}|X^{(K)}_t) - h(X^{(I)}_{t+1}|X^{(K)}_t,X^{(J)}_t)],$$ provided that the limit exists . Since, in general, $H(X|Y)-H(X|Y,Z)=I(X;Z|Y)$, where the latter defines the conditional mutual information between $X$ and $Z$ conditioned on $Y$, it follows from the nonnegativity of conditional mutual information that CSE is nonnegative. When $K=\varnothing$, we omit the conditioning part and simply use the notation $C_{J\rightarrow I}$. Notice that if $K=I$, CSE reduces to TE. On the other hand, CSE generalizes TE by allowing $K$ to be an arbitrary set of components.

## Optimal Causation Entropy Principle

In our recent work , we revealed the equivalence between the problem of identifying the causal components and the optimization of CSE. In particular, we proved that for an arbitrary component $i$, its unique set of causal components $N_i$ is the *minimal set of components that maximizes CSE*. Given the collection of sets (of components) with maximal CSE with respect to component $i$, $$\label{eq:optimalcse1}
    \mathcal{K}_i=\{K\subset\mathcal{V}|C_{K\rightarrow i}\geq C_{K'\rightarrow i}~\mbox{for any}~K'\subset\mathcal{V} \},\vspace{-3pt}$$`<!-- -->`{=html} it was shown that $N_i$ is the unique set in $\mathcal{K}_i$ with minimal cardinality, *i.e.*, $$\label{eq:optimalcse2}
    N_i = \cap_{K\in\mathcal{K}_i}K = {\operatorname{argmin}}_{K\in\mathcal{K}_i}|K|.$$ We refer to this minimax principle as the oCSE principle .

## Computational Causality Inference

Based on the oCSE principle, we developed two algorithms whose joint sequential application allows the inference of causal relationships within a system . The goal of these algorithms is to effectively and efficiently identify for a given component $i$ the direct causal set of components $N_i$. The first algorithm aggregatively identifies the potential causal components of $i$. The outcome is a set of components $M_i$ that includes $N_i$ as its subset, with possibly additional components. These additional components are then progressively removed by applying the second algorithm.

For a given component $i$, the first algorithm, referred to as aggregative discovery, starts by selecting a component $k_1$ that maximizes CSE, *i.e.*, $$k_1={\operatorname{argmax}}_{k\in\mathcal{V}}C_{k\rightarrow i}.$$ Then, at each step $j$ ($j=1,2,\dots$), a new component $k_{j+1}$ is identified among the rest of the components to maximize the CSE conditioned on the previously selected components: $$k_{j+1}=\underset{k\in\mathcal{V}/\{k_1,k_2,\dots,k_j\}}{\operatorname{argmax}}C_{k\rightarrow i|\{k_1,k_2,\dots,k_j\}}.
%k_{j+1}={\operatorname{argmax}}_{k\in\mathcal{V}-\{k_1,k_2,\dots,k_j\}}C_{k\rightarrow i|\{k_1,k_2,\dots,k_j\}}.$$ Recall that CSE is nonnegative. The above iterative process is terminated when the corresponding maximum CSE equals zero, *i.e.*, when: $$\label{eq:maxcsezero}
    \underset{k\in\mathcal{V}/\{k_1,k_2,\dots,k_j\}}\max C_{k\rightarrow i|\{k_1,k_2,\dots,k_j\}} = 0,$$ and the outcome is the set of components $M_i=\{k_1,k_2,\dots,k_j\}\supset N_i$.

Next, to remove non-causal components (including indirect and spurious ones) that are in $M_i$, but not in $N_i$, we employ the second algorithm, referred to as progressive removal. A component $k_j$ in $M_i$ is removed when: $$\label{eq:csezero}
    C_{k_j\rightarrow i|M_i/\{k_j\}}=0,$$ and $M_i$ is updated accordingly[^1]. After removing all such components, the resulting set is identified as $N_i$.

## Estimation of CSE in Practice

In practice, causation entropy $C_{J\rightarrow I|K}$ needs to be *estimated* from data. We define the Gaussian estimator of causation entropy as: $$\label{eq:cseest}
    \widehat{C}^{\mbox{\scriptsize (Gaussian)}}_{J\rightarrow I|K} \equiv \frac{1}{2}\log
    \left(\frac{\operatorname{det}\left[\widehat\Phi(0)_{II}-\widehat\Phi(1)_{IK}\widehat\Phi(0)_{KK}^{-1}\widehat\Phi(1)_{IK}^\top\right]}
    {\operatorname{det}\left[\widehat\Phi(0)_{II}-\widehat\Phi(1)_{I,K\cup J}\widehat\Phi(0)_{K\cup J,K\cup J}^{-1}\widehat\Phi(1)_{I,K\cup J}^\top\right]}\right).$$ Here: $$\Phi(\tau)_{IJ}\equiv\operatornamewithlimits{Cov}(X^{(I)}_{t+\tau},X^{(J)}_t)\vspace{3pt}$$ denotes a covariance matrix where $\tau\in\{0,1\}$, and $\widehat\Phi(\tau)_{IJ}$ denotes the corresponding sample covariance matrix estimated from the data . The estimation $\widehat{C}^{\mbox{\scriptsize (Gaussian)}}_{J\rightarrow I|K}\approx C_{J\rightarrow I|K}$ if: (i) the underlying random variables are Gaussians; and (ii) the amount of data is sufficient for the relevant sample covariances to be close to their true values. When the underlying random variables are non-Gaussian, an efficient estimator for CSE is yet to be fully developed. As previously pointed out, binning-related estimators, although conceptually simple, are generally inefficient, unless the sample space is low-dimensional . To gain efficiency and to minimize bias for a large system where the sample space is high-dimensional, an idea would be to build upon the $k$-nearest neighbor estimators of entropic measures .

Regardless of the method that is being adopted for the estimation of $C_{J\rightarrow I|K}$, the estimated value is unlikely to be exactly zero due to limited data and numerical precision. In practice, it is necessary to decide whether or not the estimated quantity should be regarded as zero. Such a decision impacts the termination criterion Equation  in the aggregative discovery algorithm and determines which components need to be removed based on Equation  in the progressive removal algorithm. As discussed in , such a decision problem can be addressed via a nonparametric statistical test, called the permutation test, as described below.

For given time series data $\{x^{(i)}_t\}$ ($i=1,2,\dots,n;~t=1,2,\dots,T$), let $\widehat{C}_{J\rightarrow I|K}$ denote the estimated value of $C_{J\rightarrow I|K}$. Based on the set of components $J$ and a permutation function $\pi$ on the set of integers from one to $T$, the corresponding permuted time series $\{y^{(i)}_t\}$ is defined as: $$y^{(i)}_t=
    \begin{cases}
    x^{(i)}_{\pi(t)} & \mbox{if $i\in J$,}\\
    x^{(i)}_t & \mbox{if $i\notin J$}.
    \end{cases}$$ To apply the permutation test, we generate a number of randomly permuted time series (the number will be denoted by $r$). We then compute the causation entropy from $J$ to $I$ conditioned on $K$ for each permuted time series to obtain $r$ values of the estimates, which are consequently used to construct an empirical cumulative distribution $F$ as: $$F(C)\equiv\frac{1}{r}\big|\{\widehat{C}^{(s)}_{J\rightarrow I|K}:\widehat{C}^{(s)}_{J\rightarrow I|K}\leq C,~1\leq s\leq r\}\big|,$$ where $\widehat{C}^{(s)}_{J\rightarrow I|K}$ are estimates from the permuted time series with $1\leq s\leq r$ and $|\cdot|$ denotes the cardinality of a set. Finally, with the null hypothesis that $C_{J\rightarrow I|K}=0$, we regard $C_{J\rightarrow I|K}$ as strictly positive if and only if: $$F(\widehat{C}_{J\rightarrow I|K})>\theta,\vspace{-3pt}$$ where $0<(1-\theta)<1$ is the prescribed significance level. In other words, the null hypothesis is rejected at level $(1-\theta)$ if the above inequality holds. Therefore, the permutation test relies on two input parameters: (i) the number of random permutations $r$; and (ii) the significance level $(1-\theta)$. In practice, the increase of $r$ improves the accuracy of the empirical distribution at the expense of additional computational cost. A reasonable balance is often achieved when $10^{3}\lesssim r\lesssim10^4$ . The value of $\theta$ sets a lower bound on the false positive ratio and should be chosen to be close to one (for example, $\theta=99\%$) .

# Extracting Stochastic Time Series from Deterministic Orbits

Biological systems can exhibit both regular and chaotic features . For instance, a healthy cardiac rhythm is characterized by a chaotic time series, whereas a pathological rhythm often exhibits regular dynamics . Therefore, a decrease of the chaoticity of the cardiac rhythm is an alarming clinical signature. A similar connection between the regularity of the time series and pathology is observed in spontaneous neuronal bursting in the brain . When a system settles into a periodic or equilibrium state, it becomes nearly impossible to infer the coupling structure among the variables, as the system is not generating any information to be utilized for inference. To overcome this difficulty, we propose to apply small stochastic perturbations to the system while in an equilibrium state (this is equivalent to adding dynamic noise to a system to facilitate coupling inference, as shown in ). Then, we measure the system's response over short time intervals. Finally, we follow the oCSE principle and apply the aggregative discovery and progressive removal algorithms to the measured data to infer the couplings among the variables.

## Dynamical System and Equilibrium States

Consider a continuous dynamical system: $$\label{eq:sys1}
    dx/dt = f(x),$$ where $x=[x_1,x_2,\dots,x_n]^\top\in\mathbb{R}^n$ is the $n$-dimensional state variable, the symbol $``\top"$ denotes transpose, and $f=[f_1,f_2,\dots,f_n]^\top:\mathbb{R}^n\rightarrow\mathbb{R}^n$ is a smooth vector field, which models the dynamic rules of the system. A trajectory of the system  is a solution $x(t)$ to the differential equation, Equation , with a given initial condition $x(0)=x_0$.

An equilibrium of the system is a state $x^*$, such that $f(x^*)=0$. When a system reaches an equilibrium, the time evolution of the state ceases. An equilibrium $x^*$ is called stable if nearby trajectories approach $x^*$ forward in time, *i.e.*, there exists a constant $\rho>0$, such that $x(t)\xrightarrow{t\rightarrow\infty}x^*$ whenever $\|x_0-x^*\|<\rho$, where $\|\cdot\|$ denotes the standard Euclidean norm. Otherwise, the equilibrium is called unstable.

## Response of the System to Stochastic Perturbations

To gain information about the coupling structure of a system, it is necessary to apply external perturbations to "knock" the system out of an equilibrium state and observe how it responds to these perturbations. Suppose that we apply and record a sequence of random perturbations to the system in such a manner that before each perturbation, the system is given sufficient time to evolve back to its stable equilibrium. In addition, the response of the system is measured shortly after each perturbation, but before the system reaches the equilibrium again. Denote the stable equilibrium of interest as $x^*$; we propose to repeatedly apply the following steps.

Step 1. Allow the system to (spontaneously) reach $x^*$.

Step 2. At time $t$, apply and record a random perturbation $\xi$ to the system, *i.e.*, $x(t)=x^*+\xi$.

Step 3. At time $t+\Delta{t}$, measure the rate of response, defined as $\eta=[x(t+\Delta{t})-x^*-\xi]/\Delta{t}$.

Repeated application of these steps $L$ times results in a collection of perturbations, denoted as $\{\xi_\ell\}$, and rates of response, denoted as $\{\eta_\ell\}$, where $\ell=1,2,\dots,L$. Here, each perturbation $\xi_\ell$ is assumed to be drawn independently from the same multivariate Gaussian distribution with zero mean and covariance matrix $\sigma^2\mathbb{I}$. For a given equilibrium $x^*$, in addition to $L$—the number of times the perturbations are applied—there are two more parameters in the stochastic perturbation process: the sample frequency defined as $1/\Delta{t}$ and the variance of perturbation defined as $\sigma^2$. To ensure that the perturbation approximates the linearized dynamics of the system, we require that $1/\Delta{t}\gg 1$ and $\sigma\ll 1$. The influence of these parameters will be studied in the next section with a concrete example.

We remark here that the choice of Gaussian distribution is a practical rather than a conceptual one. In theory, any multivariate distribution can be used to generate the perturbation vector $\xi_\ell$ as long as the component-wise distributions of $\xi^{(i)}_\ell$ and $\xi^{(j)}_k$ are identical and independent whenever $i\neq{j}$ or $\ell\neq k$. In practice, since the effectiveness of any information theoretic method (including the one proposed here) depends on the estimation of entropies, choosing the perturbations to be Gaussian greatly improves the reliability of estimation and, thus, the accuracy of the inference. This is because the entropy of Gaussian variables depends only on covariances, rendering its estimation relatively straightforward .

Note that each perturbation $\xi_\ell$ and its response $\eta_\ell$ are related through the underlying nonlinear differential equation $dx/dt=f(x)$, where the nonlinearity is encoded in the function $f(x)$, which is assumed to be unknown. For an equilibrium $x^*$, the dynamics of nearby trajectories can be approximated by its linearized system, as follows. Consider a state $x\approx x^*$ and define the new variable $\delta{x}=x-x^*$. To the first order, the time evolution of $\delta x$ can be approximated by the following linear system: $$\label{eq:vareq}
    d(\delta{x})/dt = Df(x^*)\delta{x},$$ where $Df(\cdot)$ is the $n$-by-$n$ Jacobian matrix of $f$, defined as $[Df(x)]_{ij}=\partial f_i/\partial x_j$. A sufficient condition for $x^*$ to be stable is that all eigenvalues of $Df(x^*)$ must have negative real parts. From Equation , with the additional assumption that $\Delta{t}\ll 1$, the relationship between the perturbation and response is approximated by the following equation: $$\label{eq:linear1}
    \eta_\ell= Df(x^*)\xi_\ell.$$ Note that since $\xi_\ell$ is a multivariate normal random variable and $Df(x^*)$ is a constant matrix, the variable $\eta_\ell$ is also (approximately) a multivariate normal random variable. Equation  therefore represents a drive-response type of Gaussian process.

# Application to Synthetic Biology

## The Repressilator

Cellular dynamics is centrally important in biology . To describe and, to a certain extent, to understand what happens in cells, fluctuating chemical concentrations and reaction rates need to be measured experimentally. However, constructing dynamic models that accurately reproduce the observed phenomena in cells is quite challenging. An alternative approach consists in engineering synthetic biological systems that follow prescribed dynamic rules. An important example is the so-called repressilator (or repression-driven oscillator) presented in . The repressilator is based upon three transcriptional repressors inserted into the *E. coli* bacteria with a plasmid. The three repressors, $lacl$, $tetR$ and $cl$, are related as follows: $lacl$ inhibits the transcription of the gene coding for $tetR$; $tetR$ inhibits the transcription of the gene coding for $cl$; $cl$ inhibits the transcription of the gene coding for $lacl$. In the absence of inhibition, each of the three proteins reaches a steady-state concentration resulting from a balance between its production and degradation rates within the bacterium. In the presence of cross-inhibition by the other two repressors, this network architecture potentially allows oscillations and other interesting dynamical behaviors and serves as a prototype of modeling the quorum sensing among bacteria species .

The repressilator dynamics can be modeled by a system of coupled differential equations, which describe the rates of change for the concentration $p_{i}$ of each protein repressor and the concentration $m_{i}$ of its associated mRNA in their network, as:   $$\begin{aligned}
\label{eq:repressilator}
\begin{cases}
\dot{m}_1 = -m_1 + \dfrac{\alpha}{(1+p^n_3)}+\alpha_0\\
\dot{m}_2 = -m_2 + \dfrac{\alpha}{(1+p^n_1)}+\alpha_0\\
\dot{m}_3 = -m_3 + \dfrac{\alpha}{(1+p^n_2)}+\alpha_0\\
\dot{p}_1 = -\beta(p_1-m_1)\\
\dot{p}_2 = -\beta(p_2-m_2)\\
\dot{p}_3 = -\beta(p_3-m_3)
\end{cases}
\end{aligned}$$ where $m_1$ ($p_1$), $m_2$ ($p_2$) and $m_3$ ($p_3$) represent the mRNA (protein) concentration of the genes $lacl$, $tetR$ and $cl$, respectively. See Figure a for a schematic representation of the system. Each ODE in Equation  consists of positive terms modeling the production rate and a negative term representing degradation. There are four parameters in the ODEs in Equation , namely: $\beta$ is the ratio of the protein decay rate to the mRNA decay rate; $n$ is the so-called Hill coefficient and describes the cooperativity of the binding of repressor to promoter; $\alpha _{0}$, the leakiness of the promoter, is the rate of transcription of mRNA in the presence of saturating concentration of the repressor; $\alpha _{0}+\alpha$ is the additional rate of transcription of mRNA in the absence of the inhibitor. Note that units of time and concentration in Equation  have been rescaled in order to make these equations non-dimensional .

As shown in , there is an extended region in the parameter space for which the system described in Equation  exhibits a single stable equilibrium. The Jacobian matrix at the equilibrium is: $$Df =
\begin{pmatrix}
-1 & 0 & 0 & 0 & 0 & -\frac{\alpha np_3^{n-1}}{(1+p^n_3)^2} \\
0 & -1 & 0 & -\frac{\alpha np_1^{n-1}}{(1+p^n_1)^2} & 0 & 0 \\
0 & 0 & -1 & 0 & -\frac{\alpha np_2^{n-1}}{(1+p^n_2)^2} & 0 \\
\beta & 0 & 0 & -\beta & 0 & 0 \\
0 & \beta & 0 & 0 & -\beta & 0 \\
0 & 0 & \beta & 0 & 0 & -\beta\vspace{-3pt}
 \end{pmatrix}.$$
**Problem statement:** The goal of coupling inference is to identify the location of the nonzero entries of the Jacobian through time series generated by the system near equilibrium.

## Inference of Coupling Structure via the Repressilator Dynamics

We consider the repressilator dynamics as modeled by Equation  with the parameters $n=2$, $\alpha_0=0$, $\alpha=10$ and $\beta=100$. Under this setting, the system exhibits a single stable equilibrium  at which $m_1=m_2=m_3=p_1=p_2=p_3=2$. Typical time series are shown in Figure b. After the system settles at the equilibrium, we apply the stochastic perturbations as described in Section  and obtain a time series of the perturbations $\{\xi_\ell\}$, as well as responses $\{\eta_\ell\}$. The goal is to identify, for each component (the mRNA or protein of a gene) in the system, the set of components that determine its dynamics (*i.e.*, the variables that appear on the right-hand side of each relation in Equation ).

To apply our algorithms according to the oCSE principle, we define the set of random variables $X^{(i)}_t$ ($i=1,2,\dots,12; t=1,2,\dots$) as: $$\label{eq:maptoX}
X^{(i)}_t=
\begin{cases}
\xi^{(i)}_t & \mbox{if $1\leq i\leq 6$},\\
\eta^{(i-6)}_{t-1} & \mbox{if $7\leq i\leq 12$}.
\end{cases}$$ The approximate relationship between the perturbation and response as in Equation  can be expressed as: $$\label{eq:linear2}
    X^{(I)}_{t+1} = AX^{(J)}_t,\vspace{3pt}$$ where $I=\{7,8,\dots,12\}$, $J=\{1,2,\dots,6\}$ and $A=Df(x^*)$. Since this equation defines a stochastic process that satisfies the Markov assumptions in Equation , the oCSE principle applies. This implies that the direct couplings can be correctly inferred, at least in principle, by jointly applying the aggregative discovery and progressive removal algorithms.

Since the perturbations $\{\xi_\ell\}$ and responses $\{\eta_\ell\}$ depend on the number of samples $L$, the rate of perturbation $1/\Delta{t}$ and the variance of perturbation $\sigma^2$, so is the accuracy of the inference. Next, we explore the change in performance of our approach by varying these parameters. We use two quantities to measure the accuracy of the inferred coupling structure, namely, the false positive ratio $\varepsilon_{+}$ and the false negative ratio $\varepsilon_{-}$. Since our goal is to infer the structure rather than the weights of the couplings, we focus on the structure of the Jacobian matrix $Df(x^*)$, encoded in the binary matrix $B$, where: $$[B]_{ij}=
\begin{cases}
1, & \mbox{if $[Df(x^*)]_{ij}\neq 0$},\\
0, & \mbox{otherwise}.
\end{cases}$$ On the other hand, applying the oCSE principle, the inferred direct coupling structure gives rise to the estimated binary matrix $\widehat{B}$, where: $$[\widehat{B}]_{ij}=
\begin{cases}
1, & \mbox{if $j$ is a direct causal component of $i$},\\
0, & \mbox{otherwise}.\vspace{-6pt}
\end{cases}$$ Given matrices $B$ and $\widehat{B}$, the false positive and false negative ratios are defined, respectively, as: $$\label{eq:inferror}
\begin{cases}
\varepsilon_{+}\equiv \dfrac{\mbox{number of $(i,j)$ pairs with $\widehat{B}_{ij}=1$ and $B_{ij}=0$}}{\mbox{number of $(i,j)$ pairs with $B_{ij}=0$}},\vspace{0.1in}\\
\varepsilon_{-}\equiv \dfrac{\mbox{number of $(i,j)$ pairs with $\widehat{B}_{ij}=0$ and $B_{ij}=1$}}
{\mbox{number of $(i,j)$ pairs with $B_{ij}=1$}}.
\end{cases}$$ It follows that $0\leq\varepsilon_{+},\varepsilon_{-}\leq1$ and $\varepsilon_{+}=\varepsilon_{-}=0$ when exact (error-free) inference is achieved.

Figure a,b shows that both the false positives and false negatives converge as the number of samples increases. However, they converge to zero only if the rate of perturbation is sufficiently high. Figure c,d supports this observation and, in addition, shows that in the high rate of perturbation regime, exact inference is achieved with a sufficient number of samples (in this case, $L\sim 100$). The combined effects of $L$ and $1/\Delta{t}$ are shown in Figure . In all simulations, the variation of the perturbation is set to be $\sigma^2=10^{-4}$ and is found to have little effect on the resulting inference, provided that it is sufficiently small to keep the linearization in Equations  and valid.

# Discussion and Conclusions

## Results Summary

In this paper, we considered the challenging problem of inferring the causal structure of complex systems from limited available data (enjoy Figure  for a cartoon depiction of the concept of causality during a soccer game). Specifically, we presented an application of the so-called oCSE principle to identify the coupling structure of a synthetic biological system, the repressilator. First, we briefly reviewed the main points of the oCSE principle (Equations  and ), which states that for an arbitrary component $i$ of a complex system, its unique set of causal components $N_{i}$ is the minimal set of components that maximizes CSE (causation entropy, defined in Equation  is a generalization of transfer entropy). We strengthen in this work our claim that CSE is a suitable information-theoretic measure for reliable statistical inferences, since it takes into account both direct and indirect influences that appear in the form of information flows between nodes of networks underlying complex systems. We also devoted some attention to the implementation of the oCSE principle. This task is accomplished by means of the joint sequential application of two algorithms, aggregative discovery and progressive removal, respectively. Second, having introduced the main theoretical and computational frameworks, we used a stochastic perturbation approach to extract time series data approximating a Gaussian process when the model system—the repressilator (see Equation )—reaches an equilibrium configuration. We then applied the above-mentioned algorithms implementing the oCSE principle in order to infer the coupling structure of the model system from the observed data. Finally, we numerically showed that the success rate of our causal entropic inferences not only improves with the amount of available measured data (Figure a), but it also increases with a higher frequency of sampling (Figure b).

One especially important feature of our oCSE-based causality inference approach is that it is immune (in principle) to false positives. When data is sufficient, false positives are eliminated by sequential joint application of the aggregative discovery and progressive removal algorithms, as well as raising the threshold $\theta$ used in the permutation test (see for a more detailed investigation of this point). In contrast, any pairwise causality measure without additional appropriate conditioning will in principle be susceptible to false positives, result in too many connections and, sometimes, even, implies that everything causes everything. Such a limitation is intrinsic and cannot be overcome merely by gathering more data . On the other hand, false negatives are less common in practice and are usually caused by ineffective statistical estimation orinsufficient data.

## oCSE for General Stochastic Processes

We presented the oCSE principle for stochastic processes under the Markov assumptions stated in Equation . From an inference point of view, each and every one of these assumptions has a well-defined interpretation. For instance, the temporal Markov condition implies the stationarity of causal dependences between nodes. The loss of temporal stationarity could be addressed by partitioning the time series data into stationary segments and, then, performing time-dependent inferences. Clearly, such inferences would require extra care. Furthermore, the loss of the spatially and/or faithfully Markov condition would imply that the set of nodes that directly influence any given node of the network describing the complex system is no longer minimal and unique. Causality inference in this case becomes an ill-posed problem. These issues will be formally addressed in a forthcoming work.

Note that a finite $k$-th order Markov process $\{X_t\}$ can always be converted into a first-order (memoryless) one $\{Z_t\}$ by lifting, or in other words, defining new random variables $Z_t=(X_{t-k+1},X_{t-k+2},\dots,X_t)$ . In this regard, the oCSE principle extends naturally to an arbitrary finite-order Markov process. On the other hand, for a general stationary stochastic process that is not necessarily Markov (*i.e.*, considering a process with potentially infinite memory), there might exist an infinite number of components from the past that causally influence the current state of a given component. However, under an assumption of vanishing (or fading) memory, such influences decay rapidly as a function of the time lag, and consequently, the process itself can be approximated by a finite-order Markov chain . We plan to leave such generalizations for forthcoming investigations.

**Acknowledgments** We thank Samuel Stanton from the Army Research Office (ARO) Complex Dynamics and Systems Program for his ongoing and continuous support. This work was funded by ARO Grant No. W911NF-12-1-0276.

**Author Contributions** Jie Sun and Erik M. Bollt designed and supervised the research. Jie Sun and Carlo Cafaro performed the analytical calculations and the numerical simulations. All authors contributed to the writing of the paper.

**Conflicts of Interest** The authors declare no conflict of interest.

[^1]: An alternative way of removing non-causal components is to test conditioning on subsets of $M_i$ with increasing cardinality, potentially reducing the dimensionality of sample space at the expense of increased computational burden. This is similar to the PC-algorithm originally proposed in Ref.  and successfully employed in Ref. , with the key difference that here the enumeration of conditioning subsets only needs to be performed for $M_i$ instead of for the entire system $\mathcal{V}$.


abstract: These pages provide you with an example of the layout and style for 100% reproduction which we wish you to adopt during the preparation of your paper. This is the output from the LaTeX document class you requested.
address: Mathematics and Computer Science Section, Elsevier Science B.V.,
P.O. Box 103, 1000 AC Amsterdam, The Netherlands; Economics Department, University of Winchester,
2 Finch Road, Winchester, Hampshire P3L T19, United Kingdom
author: P. de Groot
[^1], R. de Maas[^2], X.-Y. Wang
and A. Sheffield[^3]
title: Elsevier instructions for the preparation of a 2-column format camera-ready paper in LaTeX

# FORMAT

Text should be produced within the dimensions shown on these pages: each column 7.5 cm wide with 1 cm middle margin, total width of 16 cm and a maximum length of 19.5 cm on first pages and 21 cm on second and following pages. The LaTeX document class uses the maximum stipulated length apart from the following two exceptions (i) LaTeX does not begin a new section directly at the bottom of a page, but transfers the heading to the top of the next page; (ii) LaTeX never (well, hardly ever) exceeds the length of the text area in order to complete a section of text or a paragraph. Here are some references: .

## Spacing

We normally recommend the use of 1.0 (single) line spacing. However, when typing complicated mathematical text LaTeX automatically increases the space between text lines in order to prevent sub- and superscript fonts overlapping one another and making your printed matter illegible.

## Fonts

These instructions have been produced using a 10 point Computer Modern Roman. Other recommended fonts are 10 point Times Roman, New Century Schoolbook, Bookman Light and Palatino.

# PRINTOUT

The most suitable printer is a laser or an inkjet printer. A dot matrix printer should only be used if it possesses an 18 or 24 pin printhead ("letter-quality").

The printout submitted should be an original; a photocopy is not acceptable. Please make use of good quality plain white A4 (or US Letter) paper size. *The dimensions shown here should be strictly adhered to: do not make changes to these dimensions, which are determined by the document class*. The document class leaves at least 3 cm at the top of the page before the head, which contains the page number.

Printers sometimes produce text which contains light and dark streaks, or has considerable lighting variation either between left-hand and right-hand margins or between text heads and bottoms. To achieve optimal reproduction quality, the contrast of text lettering must be uniform, sharp and dark over the whole page and throughout the article.

If corrections are made to the text, print completely new replacement pages. The contrast on these pages should be consistent with the rest of the paper as should text dimensions and font sizes.

# TABLES AND ILLUSTRATIONS

Tables should be made with LaTeX; illustrations should be originals or sharp prints. They should be arranged throughout the text and preferably be included *on the same page as they are first discussed*. They should have a self-contained caption and be positioned in flush-left alignment with the text margin within the column. If they do not fit into one column they may be placed across both columns (using `\begin{table*}` or `\begin{figure*}` so that they appear at the top of a page).

## Tables

Tables should be presented in the form shown in Table . Their layout should be consistent throughout.

```latex
\begin{table*}[htb]\caption{The next-to-leading order (NLO) results
{\em without} the pion field.}
\label{table:1}
\newcommand{\m}{\hphantom{$-$}}
\newcommand{\cc}[1]{\multicolumn{1}{c}{#1}}
\renewcommand{\tabcolsep}{2pc} % enlarge column spacing
\renewcommand{\arraystretch}{1.2} % enlarge line spacing
\begin{tabular}{@{}lllll}
\hline
$\Lambda$ (MeV)           & \multicolumn{1}{c}{$140$} & \multicolumn{1}{c}{$150$} & \multicolumn{1}{c}{$175$} & \multicolumn{1}{c}{$200$} \\
\hline
$r_d$ (fm)                & \hphantom{$-$}1.973 & \hphantom{$-$}1.972 & \hphantom{$-$}1.974 & \hphantom{$-$}1.978 \\
$Q_d$ ($\mbox{fm}^2$)     & \hphantom{$-$}0.259 & \hphantom{$-$}0.268 & \hphantom{$-$}0.287 & \hphantom{$-$}0.302 \\
$P_D$ (\%)                & \hphantom{$-$}2.32  & \hphantom{$-$}2.83  & \hphantom{$-$}4.34  & \hphantom{$-$}6.14  \\
$\mu_d$                   & \hphantom{$-$}0.867 & \hphantom{$-$}0.864 & \hphantom{$-$}0.855 & \hphantom{$-$}0.845 \\
$\mathcal{M}_{\mathrm{M1}}$ (fm)   & \hphantom{$-$}3.995 & \hphantom{$-$}3.989 & \hphantom{$-$}3.973 & \hphantom{$-$}3.955 \\
$\mathcal{M}_{\mathrm{GT}}$ (fm)   & \hphantom{$-$}4.887 & \hphantom{$-$}4.881 & \hphantom{$-$}4.864 & \hphantom{$-$}4.846 \\
$\delta_{\mathrm{1B}}^{\mathrm{VP}}$ (\%)
                          & $-0.45$ & $-0.45$ & $-0.45$ & $-0.45$ \\
$\delta_{\mathrm{1B}}^{\mathrm{C2:C}}$ (\%)
                          & \hphantom{$-$}0.03  & \hphantom{$-$}0.03  & \hphantom{$-$}0.03  & \hphantom{$-$}0.03  \\
$\delta_{\mathrm{1B}}^{\mathrm{C2:N}}$ (\%)
                          & $-0.19$ & $-0.19$ & $-0.18$ & $-0.15$ \\
\hline
\end{tabular}\\[2pt]
The experimental values are given in ref. \cite{Eato75}.
\end{table*}
```
```latex
\begin{sidewaystable}
\caption{The next-to-leading order (NLO) results
{\em without} the pion field.}
\label{table:2}
\newcommand{\m}{\hphantom{$-$}}
\newcommand{\cc}[1]{\multicolumn{1}{c}{#1}}
\renewcommand{\arraystretch}{1.2} % enlarge line spacing
\begin{tabular*}{\textheight}{@{\extracolsep{\fill}}lllllllllllll}
\hline
& $\Lambda$ (MeV) & \multicolumn{1}{c}{$140$} & \multicolumn{1}{c}{$150$} & \multicolumn{1}{c}{$175$} & \multicolumn{1}{c}{$200$} & \multicolumn{1}{c}{$225$} & \multicolumn{1}{c}{$250$} &
\multicolumn{1}{c}{Exp.} & \multicolumn{1}{c}{$v_{18}$~\cite{v18}} &  \\
\hline
%b
 & $r_d$ (fm)                        & \hphantom{$-$}1.973 & \hphantom{$-$}1.972 & \hphantom{$-$}1.974 & \hphantom{$-$}1.978 & \hphantom{$-$}1.983 & \hphantom{$-$}1.987 & 1.966(7) & \hphantom{$-$}1.967 & \\[2pt]
 & $Q_d$ ($\mbox{fm}^2$)             & \hphantom{$-$}0.259 & \hphantom{$-$}0.268 & \hphantom{$-$}0.287 & \hphantom{$-$}0.302 & \hphantom{$-$}0.312 & \hphantom{$-$}0.319 & 0.286    & \hphantom{$-$}0.270 & \\[2pt]
 & $P_D$ (\%)                        & \hphantom{$-$}2.32  & \hphantom{$-$}2.83  & \hphantom{$-$}4.34  & \hphantom{$-$}6.14  & \hphantom{$-$}8.09  & \hphantom{$-$}9.90  & $-$      & \hphantom{$-$}5.76  & \\[2pt]
 & $\mu_d$                           & \hphantom{$-$}0.867 & \hphantom{$-$}0.864 & \hphantom{$-$}0.855 & \hphantom{$-$}0.845 & \hphantom{$-$}0.834 & \hphantom{$-$}0.823 & 0.8574   & \hphantom{$-$}0.847 & \\[5pt]
 & $\mathcal{M}_{\mathrm{M1}}$ (fm)             & \hphantom{$-$}3.995 & \hphantom{$-$}3.989 & \hphantom{$-$}3.973 & \hphantom{$-$}3.955 & \hphantom{$-$}3.936 & \hphantom{$-$}3.918 & $-$      & \hphantom{$-$}3.979 & \\[5pt]
 & $\mathcal{M}_{\mathrm{GT}}$ (fm)             & \hphantom{$-$}4.887 & \hphantom{$-$}4.881 & \hphantom{$-$}4.864 & \hphantom{$-$}4.846 & \hphantom{$-$}4.827 & \hphantom{$-$}4.810 & $-$      & \hphantom{$-$}4.859 & \\[2pt]
 & $\delta_{\mathrm{1B}}^{\mathrm{VP}}$ (\%)   & $-0.45$ & $-0.45$ & $-0.45$ & $-0.45$ & $-0.45$ & $-0.44$ & $-$      & $-0.45$ & \\[2pt]
 & $\delta_{\mathrm{1B}}^{\mathrm{C2:C}}$ (\%) & \hphantom{$-$}0.03  & \hphantom{$-$}0.03  & \hphantom{$-$}0.03  & \hphantom{$-$}0.03  & \hphantom{$-$}0.03  & \hphantom{$-$}0.03  & $-$      & \hphantom{$-$}0.03  & \\[2pt]
 & $\delta_{\mathrm{1B}}^{\mathrm{C2:N}}$ (\%) & $-0.19$ & $-0.19$ & $-0.18$ & $-0.15$ & $-0.12$ & $-0.10$ & $-$      & $-0.21$ & \\
\hline
\end{tabular*}\\[2pt]
The experimental values are given in ref. \cite{Eato75}.
\end{sidewaystable}
```

Horizontal lines should be placed above and below table headings, above the subheadings and at the end of the table above any notes. Vertical lines should be avoided.

If a table is too long to fit onto one page, the table number and headings should be repeated above the continuation of the table. For this you have to reset the table counter with `\addtocounter{table}{-1}`. Alternatively, the table can be turned by $90^\circ$ ('landscape mode') and spread over two consecutive pages (first an even-numbered, then an odd-numbered one) created by means of `\begin{table}[h]` without a caption. To do this, you prepare the table as a separate LaTeX document and attach the tables to the empty pages with a few spots of suitable glue.

## Useful table packages

Modern LaTeX comes with several packages for tables that provide additional functionality. Below we mention a few. See the documentation of the individual packages for more details. The packages can be found in LaTeX's `tools` directory.

`array`

:   Various extensions to LaTeX's `array` and `tabular` environments.

`longtable`

:   Automatically break tables over several pages. Put the table in the `longtable` environment instead of the `table` environment.

`dcolumn`

:   Define your own type of column. Among others, this is one way to obtain alignment on the decimal point.

`tabularx`

:   Smart column width calculation within a specified table width.

`rotating`

:   Print a page with a wide table or figure in landscape orientation using the `sidewaystable` or `sidewaysfigure` environments, and many other rotating tricks. Use the package with the `figuresright` option to make all tables and figures rotate in clockwise. Use the starred form of the `sideways` environments to obtain full-width tables or figures in a two-column article.

## Line drawings

Line drawings may consist of laser-printed graphics or professionally drawn figures attached to the manuscript page. All figures should be clearly displayed by leaving at least one line of spacing above and below them. When placing a figure at the top of a page, the top of the figure should align with the bottom of the first text line of the other column.

Do not use too light or too dark shading in your figures; too dark a shading may become too dense while a very light shading made of tiny points may fade away during reproduction.

All notations and lettering should be no less than 2 mm high. The use of heavy black, bold lettering should be avoided as this will look unpleasantly dark when printed.

## PostScript figures

Instead of providing separate drawings or prints of the figures you may also use PostScript files which are included into your LaTeX file and printed together with the text. Use one of the packages from LaTeX's `graphics` directory: `graphics`, `graphicx` or `epsfig`, with the `\usepackage` command, and then use the appropriate commands (`\includegraphics` or `\epsfig`) to include your PostScript file.

The simplest command is: `\includegraphics{file}`, which inserts the PostScript file `file` at its own size. The starred version of this command: `\includegraphics*{file}`, does the same, but clips the figure to its bounding box.

With the `graphicx` package one may specify a series of options as a key–value list, e.g.:

|                                               |
|:----------------------------------------------|
| `\includegraphics[width=15pc]{file}`          |
| `\includegraphics[height=5pc]{file}`          |
| `\includegraphics[scale=0.6]{file}`           |
| `\includegraphics[angle=90,width=20pc]{file}` |

See the file `grfguide`, section "Including Graphics Files", of the `graphics` distribution for all options and a detailed description.

The `epsfig` package mimicks the commands familiar from the package with the same name in LaTeX2.09. A PostScript file `file` is included with the command `\psfig{file=file}`.

Grey-scale and colour photographs cannot be included in this way, since reproduction from the printed CRC article would give insufficient typographical quality. See the following subsections.

## Black and white photographs

Photographs must always be sharp originals (*not screened versions*) and rich in contrast. They will undergo the same reduction as the text and should be pasted on your page in the same way as line drawings.

## Colour photographs

Sharp originals (*not transparencies or slides*) should be submitted close to the size expected in publication. Charges for the processing and printing of colour will be passed on to the author(s) of the paper. As costs involved are per page, care should be taken in the selection of size and shape so that two or more illustrations may be fitted together on one page. Please contact the Author Support Department at Elsevier (E-mail: `firstname.lastname@example.com`) for a price quotation and layout instructions before producing your paper in its final form.

# EQUATIONS

Equations should be flush-left with the text margin; LaTeX ensures that the equation is preceded and followed by one line of white space. LaTeX provides the document class option `fleqn` to get the flush-left effect.

$$H_{\alpha\beta}(\omega) = E_\alpha^{(0)}(\omega) \delta_{\alpha\beta} +
                          \langle \alpha | W_\pi | \beta \rangle$$

You need not put in equation numbers, since this is taken care of automatically. The equation numbers are always consecutive and are printed in parentheses flush with the right-hand margin of the text and level with the last line of the equation. For multi-line equations, use the `eqnarray` environment.

For complex mathematics, use the A-.1667em.5ex-.125emSmath package. This package sets the math indentation to a positive value. To keep the equations flush left, either load the `espcrc` package *after* the A-.1667em.5ex-.125emSmath package or set the command `\mathindent=0pt` in the preamble of your article.

References should be collected at the end of your paper. Do not begin them on a new page unless this is absolutely necessary. They should be prepared according to the sequential numeric system making sure that all material mentioned is generally available to the reader. Use `\cite` to refer to the entries in the bibliography so that your accumulated list corresponds to the citations made in the text body.

Above we have listed some references according to the sequential numeric system .

[^1]: Footnotes should appear on the first page only to indicate your present address (if different from your normal address), research grant, sponsoring agency, etc. These are obtained with the `'134thanks` command.

[^2]: For following authors with the same address use the `'134addressmark` command.

[^3]: To reuse an addressmark later on, label the address with an optional argument to the `'134address` command, e.g. `'134 address[MCSD]`, and repeat the label as the optional argument to the `'134addressmark` command, e.g. `'134addressmark[MCSD]`.


abstract: We have developed a unified format for *phylogenetic placements*, that is, mappings of environmental sequence data (e.g. short reads) into a phylogenetic tree. We are motivated to do so by the growing number of tools for computing and post-processing phylogenetic placements, and the lack of an established standard for storing them. The format is lightweight, versatile, extensible, and is based on the JSON format which can be parsed by most modern programming languages. Our format is already implemented in several tools for computing and post-processing parsimony- and likelihood-based phylogenetic placements, and has worked well in practice. We believe that establishing a standard format for analyzing read placements at this early stage will lead to a more efficient development of powerful and portable post-analysis tools for the growing applications of phylogenetic placement.
author: Frederick A Matsen; Noah G Hoffman; Aaron Gallagher; Alexandros Stamatakis
bibliography: placefmt.bib
title: A format for phylogenetic placements

# Introduction

"Phylogenetic placement" has become popular in the last several years as a way to gain an evolutionary understanding of a large collection of sequences. The input to a phylogenetic placement algorithm consists of a reference tree, a corresponding reference multiple sequence alignment, and a collection of query sequences. The output of a phylogenetic placement algorithm is a set of assignments of the query sequences to branches of the tree; there is at least one such assignment for each query. A query can be assigned to more than one branch on the reference tree to express placement uncertainty for that query sequence.

Phylogenetic placement methods circumvent several problems associated with applying traditional phylogenetic algorithms to large, environmentally-derived sequence data. The computational burden is decreased compared to constructing a tree containing reference and query sequences *de novo*, resulting in algorithms that can place thousands to tens of thousands of query sequences per hour *and* per processing core into a reference phylogeny with one thousand taxa. Because computation is performed on each query sequence individually and independently, placement algorithms are also straightforward to parallelize. The relationships between the query sequences are not investigated. Hence, the size of the search space is reduced from an exponential to just a linear number of phylogenetic hypotheses. Moreover, short and/or non-overlapping query sequences pose less of a problem, as query sequences are compared to the full-length reference sequences. Visualization of samples and comparison between samples are facilitated by the assumption of a fixed reference tree, that can be drawn in a way which highlights the location and distribution of reads.

The advent of high-throughput sequencing has motivated a growing interest in phylogenetic placement. The basic idea is as old as computational phylogenetics although these insertions historically have been considered as just the first step towards full *de novo* tree reconstruction. Recent implementations have focused on algorithms for likelihood-based placement, such as , with more efficient recent implementations . These tools are being incorporated into popular workflows for microbial ecology, such as QIIME and the next version of AMPHORA . Comparative methods are being developed and implemented in software , and work is underway to extend a tree viewer to visualize placements. Dedicated algorithms to align reads with respect to reference alignments for subsequent phylogenetic placement are also being developed .

Because of this expansion of activity, standards are needed. The original versions of pplacer  and EPA  each implemented their own idiosyncratic tabular file formats. These ad-hoc formats kept post-analysis tools from being interoperable and hindered tool comparison.

In this letter, we describe a lightweight file format that will ensure consistency between tools. Because it adopts JSON (JavaScript Object Notation) , a widely used data interchange standard, and extends the widely used Newick format for phylogenetic trees, it is straight-forward to parse using existing tools. It can be used with likelihood, posterior probability, and parsimony-based placements, can associate an arbitrary number of sequence names associated with a placement, and can store a generalization of a name list called a *named multiplicity* as described below. Basic operations such as subsetting arbitrary collections of placements and merging these lists are easily done. The format can be extended to incorporate additional information, such as taxonomic assignments.

Although we have made our best efforts to ensure that the format is sufficiently extensible without changing the specification, it may be necessary to change it in the future. For that reason, the authoritative version of the file format will be maintained on the <http://arxiv.org/> server as an online preprint of the same name. The version described in this document is version 3 of the file format.

# Concepts

We first establish terminology in order to describe the placement format. As described above, phylogenetic placement is performed by inserting a collection of query sequences onto a fixed reference tree in order to optimize a given criterion. Specifically, for a given set of query sequences the objective is to find an attachment of each query sequence to the tree that maximizes likelihood or minimizes the parsimony score for the the reference tree with that (and only that) query sequence attached. Because each query sequence is placed individually on the tree, the run time complexity is of order the product of the number of reference sequences, the number of query sequences, and the number of sites in the alignment.

There may be more than one good or likely location for a query sequence, and it is important to record this uncertainty. Uncertainty may be expressed in terms of placement locations that have equal parsimony scores, in terms of *likelihood weight ratio* (the ratio of likelihoods of the various placements), or in terms of posterior probability. Because a given query sequence thus can be considered to have a collection of placements with varying certainties, we use the word *pquery* for "placed query" to denote the collection of placements of a single query sequence.

It is also common to obtain several identical sequence reads from deep-sequencing studies. Furthermore, closely related sequences may exhibit such similar placement results that a user may wish to group them together for ease of analysis. For this reason, we allow more than one sequence name to be associated with a given pquery.

Users may simply wish to keep the number of sequences associated with a given pquery instead of the complete collection of names. More generally, they may wish to simply have a single floating point number, the *multiplicity* associated with a pquery. This multiplicity may represent a transformed measure of the quantity of sequences associated with that pquery, analogous to the transforms that are commonly applied to ecological count data. For that reason, we also allow the specification of a named multiplicity associated with a pquery in place of a list of names.

# Design

One possible representation of a collection of placements would be a single tree with each placement inserted as a pendant branch. That design is problematic for representations of uncertainty; if each possible location for every query sequence were represented as a pendant branch, then it would be difficult to distinguish the pendant edges that resulted from uncertainty with those resulting from multiple query sequences. Subsetting collections of placements would require tree "surgery". Furthermore, packing everything into a tree would make placement-specific metadata such as multiple confidence measures difficult to keep track of. Also, visualizing a reference tree with 1,000 taxa and 10,000 queries *and* with several placements per query may become computationally and visually cumbersome.

These considerations led us to develop a format where the placements are represented as a list, and their branch assignments are indexed by numbered edges of the reference tree. Each placement is associated with entries for a collection of *fields*, which can contain arbitrary data about the placement. With such a list-based format, subsetting pqueries becomes trivial.

With the separation of reference set and placements in mind, our goals in designing the format were: to adopt a popular extensible open standard human-readable file format, to ease parsing between languages and tools, and to deploy a light-weight format that can handle large collections of placements on large reference trees without requiring too much space. We chose JSON, since it satisfies all of the above criteria.

Using the JSON syntax, one option would be to individually associate each placement with an arbitrary collection of information using key-value pairs for each placement. However, doing so would have created a substantial file size overhead, as the total number of characters used to represent the keys would be about the same as the total number of characters used to represent the data. Because of this, the field titles are written out only once, and every placement just supplies the data as an array with entries in the correct corresponding order, as described below.

# Specification

Files using the format described in this paper will use the `.jplace` suffix, which is short for JSON placement.

The basic types in a JSON file are `Array`, `Boolean`, `Number`, `Object`, `String`, and `null`. These are familiar terms except `Object`, which is a list of colon separated key-value pairs, where the keys are strings and the values are arbitrary types. A JSON file contains a single JSON object.

In `.jplace` files, the fundamental object contains a list of four keys: "tree", "fields", "placements", "metadata", and "version". We will describe each of these in succession, but this need not correspond to their order in the JSON object. Indeed, the order of key-value pairs in a JSON object is unspecified.

## tree

To represent the tree, we extend the well-known Newick file format. In that format, commas and parentheses are used to display the structure of the tree. The taxon names (leaf labels) are inserted as plain text. It is also common to label internal nodes with strings appearing after the closing of a parenthesis. It is also possible to label edges of the tree with strings enclosed in square brackets. For example, the tree

    ((A:.01[e], B:.01)D:.01[g], C:.01[h]);

is a tree with some edge labels and some node labels.

We extend this format with edge numberings in curly braces:

    ((A:.01[e]{0}, B:.01{1})D:.01{3}[g], C:.01{4}[h]){5};

These edge numberings index the edges for the placements. We use curly braces to distinguish between our edge numberings and other edge label values such as posterior probability or bootstrap branch (bipartition) support.

Although not required for parsing, we use a convention that placement algorithms should use a pre-defined edge numbering. Specifically, we enforce that branches are labeled by a depth-first traversal (descending left subtree first and starting at the root/top-level node in the reference input tree) and we assign branch numbers by a post-order traversal. This strict definition is convenient to ensure one-to-one comparability of results obtained from various placement algorithms.

We also require the output tree to be identical as a planar tree to the input reference tree, that is, the subtree ordering and top-level trifurcation must remain unchanged. In the case of parsimony-based placements, the reference tree may optionally be represented without branch lengths.

## fields

The value associated with "fields" is an array of strings specifying the headers in the same order as the arrays of placement data. For example, the default fields for a maximum likelihood EPA or pplacer run are edge_num, likelihood, like_weight_ratio, distal_length, and pendant_length.

The edge_num specifies the placement edge, and is necessary for all placements. The pendant_length is the branch length for the placement edge, and distal_length is the length from the distal (away from the root) side of the reference tree edge to the placement attachment location. The likelihood is the likelihood of the tree with the placement attached, which may be calculated from an alignment with columns masked out that do not appear in the read. For that reason, the log likelihood of the placement may be better (closer to zero) than the log likelihood of the reference tree on the full-length alignment. The like_weight_ratio is the ratio of that placement's likelihood to that of the other alternate placements for that read. For a pplacer posterior probability run, the marginal likelihood marginal_prob and the posterior probability post_prob are also included.

In contrast to pplacer, EPA optimizes three branch lengths associated with a placement: the pendant branch length, the distal branch length, and the proximal branch length. Thus, the EPA output could be extended to comprise the full information generated by the EPA algorithm by adding a proximal_length field. Because the currently available downstream placement analysis tools (e.g., guppy) do not use this additional information, it is not included in the EPA `.jplace` output file at present.

The corresponding fields for parsimony-based placements (currently only available in EPA) are edge_num and parsimony. The parsimony field just contains the parsimony score of the placement as an integer.

## placements

The value associated with the "placements" key is the list of placements grouped into pqueries. The representation of each pquery is a JSON object of its own, with two keys: "p", for placements, and either "n" for names or "nm" for names with multiplicity. The value associated with "p" is the list of placements for that pquery with entries corresponding to the fields in the order set up by the "fields" described above. The list of placements shows possible placement locations along with their confidence scores and other information. The value associated with "n" is a list of names associated with that pquery. Although an arbitrary list of names can be associated with a pquery, the typical use will be to collect placement information for identical or closely related sequences. The value associated with "nm" is a list of *named multiplicities*, which as simply ordered pairs of a name and then a positive floating point value. As described above, multiplicity can be used to keep track of the number of sequences associated with that name or a transform thereof.

For parsimony-based placements we require all equally parsimonious placements of a query to be included in the output file. This is to enable easy comparison between parsimony-based placement methods; if only one of the best-scoring placements is arbitrarily selected in one way or another, comparing programs based on our standard will become error-prone and biased.

## Other keys

There are also two other keys in the fundamental JSON object. The first, "version", is mandatory, and indicates an integer version number of the format. The version described in this paper is 3. The second, "metadata", is optional and keys an arbitrary object for metadata. It can describe how the placement file was generated, which phylogenetic model was used, and so on. In EPA and pplacer we include the full command line string of the placement program invocation to allow for easy reproducability of results.

## A small example

    {
      "tree": "((A:0.2{0},B:0.09{1}):0.7{2},C:0.5{3}){4};",
      "placements":
      [
        {"p":
          [[1, -2578.16, 0.777385, 0.004132, 0.0006],
           [0, -2580.15, 0.107065, 0.000009, 0.0153]
          ],
         "n": ["fragment1", "fragment2"]
        },
        {"p": [[2, -2576.46, 1.0, 0.003555, 0.000006]],
         "nm": [["fragment3", 1.5], ["fragment4", 2]]}
      ],
      "metadata":
      {"invocation":
        "pplacer -c tiny.refpkg frags.fasta"
      },
      "version": 3,
      "fields":
      ["edge_num", "likelihood", "like_weight_ratio",
                   "distal_length", "pendant_length"]
    }

## Tabular representation

The JSON object can be readily transformed into a tabular format to more easily summarize or explore the data using statistical tools or a relational database. With the addition of an index ("placement_id") to form a relation between placements and sequence names, two tables are sufficient: one with columns "placement_id" followed by each of the fields contained by each pquery array, and another providing a mapping of every "placement_id" to each of the corresponding sequence names or named multiplicities. This transformation can be performed efficiently using any modern high level language with a JSON parsing library. Such a representation of the data is useful for supporting analyses that involve grouping and partitioning placements and sequences.

# Tools

The latest versions of EPA (<http://github.com/stamatak/standard-RAxML>) and pplacer (<http://matsen.fhcrc.org/pplacer/>) both produce these files. The guppy program in the pplacer suite has a number of subcommands that allow transformations and filterings of these files (manuscript in preparation). MePal, an implementation of placement using an alignment-free generalization to indels of Felsenstein's phylogenetic pruning algorithm , now imports and writes out this format as well. The TopiaryExplorer tree visualization package is now in the process of being extended to read this format for visualization.

# Conclusion

We have designed a unified format for phylogenetic placements. The format is lightweight, flexible, and is based on JSON, a well-established data interchange standard. The format handles placement uncertainty and allows for multiple sequence names to be associated with the placement of a single sequence. Current versions of two placement software packages have already adopted the format, and others are in the process of doing so.