Equality Constraints in Linear Hawkes Processes
Abstract
Conditional independence is often used as a testable implication of causal models of random variables.
In addition, equality constraints have been proposed to distinguish between data-generating mechanisms.
We show that one can also ﬁnd equality constraints in linear Hawkes processes, extending this theory to a class of continuous-time stochastic processes.
This is done by proving that Hawkes process models in a certain sense satisfy the equality constraints of linear structural equation models.
These results allow more reﬁned constraint-based structure learning in this class of processes.
Arguing the existence of equality constraints leads us to new identiﬁcation results for Hawkes processes.
We also describe a causal interpretation of the linear Hawkes process which is closely related to its so-called cluster representation.
1. Introduction
In causal inference, the question of what can be learned about the underlying structure from observational data is a classical one (Spirtes et al., 2000; Pearl, 2009).
While most work studies models of random variables and without an explicit notion of time, this question has also been studied for stochastic process models.
For a multivariate stochastic process, X t =
( X 1 t , X 2 t , . . .
, X nt ) T , this is often formalized by assuming that the dynamical evolution of each coordinate process, X it , depends directly only on a subset of the other coordinate processes which can be represented by a directed graph which is the structure that methods aim to learn.
As examples of this, Eichler et al.
Xu et al.
(2016) learn graphs describing the structure of multivariate linear Hawkes processes from dynamical observation of the process whereas Achab et al.
(2017) use integrated cumulants of linear
Hawkes processes for structure learning.
These approaches assume that we have full observation in the sense that every coordinate process of the system is observed.
Meek (2014); Mogensen et al.
; Mogensen (2020) consider structure learning based on tests of so-called local independence and provide methods that can recover structural information based on a marginal distribution only.
In this paper, we show that one can use equality constraints in linear Hawkes processes to distinguish between graphical structures that are indistinguishable using tests of local independence only which enables more reﬁned structure learning algorithms.
This development parallels that of structural causal models without an explicit notion of time in which equality constraints have been used for structure learning in addition to conditional independence constraints (Robins, 1986;
Verma and Pearl, 1990; Robins, 1999; Tian and Pearl, 2002; Shpitser et al., 2014; Richardson et al., 2017; Bhattacharya et al., 2021; Wang and Drton, 2020).
Some results are not proven in the main text.
Their proofs can be found in Appendix B.
Time
C oo r d i na t e p r
o
c
e ss e s
Time
2. Hawkes Processes
Hawkes processes have received much attention in machine learning in recent years as a tractable model of mutually exciting streams of events (Zhou et al., 2013; Luo et al., 2015; Etesami et al., 2016; Tan et al., 2018; Xu et al., 2018; Trouleau et al., 2019).
The following is a brief introduction to this model class, see Laub et al.
(2015), Liniger (2009), and Daley and Vere-Jones (2003)
for additional background on Hawkes processes and point processes in general.
We consider a ﬁltered probability space (Ω , F , ( F t ) , P ) and an n -dimensional point process,
X t =
( X 1 t , X 2 t , . . .
, X nt ) T , and let V = { 1 , 2 . . . , n } .
F t is a ﬁltration, that is, a nondecreasing family of σ -algebras which for each time point, t ∈ R , represent the available information at that time.
For each α ∈ V , we say that X α is a coordinate process .
Each coordinate process is described by a series of events , or points , that occur at time points indexed by R .
For α ∈ V , this is formalized by random variables, { T αi } i ∈ Z , such that T αi < T αi +1 almost surely.
For s < t , we let the N αt − N αs denote the number of points (events) in the α -process in the interval ( s, t ] , that is, N αt − N αs = (cid:80)
i 1 s<
T αi ≤ t .
We assume that no two events occur simultaneously.
For α ∈ V , the intensity process , λ α t , satisﬁes
[Equation]
The intensity, λ αt , therefore describes how likely an α -event is in the immediate future ( t, t + h ] , for small h , given the past until time t .
Deﬁnition 1 (Linear Hawkes process)
We say that a point process, X , is a linear Hawkes process if for all β ∈ V and t ∈
R the intensity is of the form
[Equation]
for a constant µ β ≥ 0 and nonnegative functions φ βα for α ∈ V .
We will often write Hawkes process instead of linear Hawkes process .
We assume throughout for all α and β that φ βα is a continuous function on (0 , ∞ ) and zero outside this interval.
We deﬁne the n × n matrix Φ such that Φ βα = (cid:82) ∞−∞ φ βα ( s ) ds.
For a square matrix, A , its spectral radius, ρ ( A ) , is the largest absolute value of its eigenvalues, λ 1 , . . .
, λ m , i.e., ρ ( A ) =
max i =1 ,...,m {| λ i |} .
If ρ ( A ) < 1 then we say that A is Schur stable .
We assume that Φ is Schur stable and therefore we can assume the Hawkes process to have stationary increments and λ t to be a stationary process (Jovanovi´c et al., 2015; Bacry and Muzy, 2016).
We deﬁne the n × n matrix R =
( I n − Φ) − 1 where
I n is the n × n identity matrix.
R is well-deﬁned due to the assumption on the spectral radius of Φ and furthermore R = (cid:80) ∞
i =
Φ i .
For a matrix, M , with rows indexed by I and columns indexed by
J , we let M ¯
I
¯ J denote the submatrix indexed by ¯ I ⊆
I and ¯ J ⊆ J .
We let dN αt denote N αt + dt − N αt where dt is the differential of t .
Following Hawkes (1971), for each α ∈ V we deﬁne
[Equation]
which is independent of t due to stationarity.
We let Λ denote the n × n diagonal matrix such that Λ αα = Λ α
and we assume Λ to be positive deﬁnite.
Integrated Covariance
We deﬁne a probabilistic concept which is seen to be a cumulative measure of covariance between two coordinate processes.
Let Σ be an n × n matrix such that for α, β ∈ V , Σ βα dt = (cid:90)
[Equation]
The entries of Σ contain the integrated covariances between pairs of coordinate processes.
The following matrix equations hold (Jovanovi´c et al., 2015; Achab et al., 2017),
[Equation]
where − T denotes transposition and inversion of a matrix.
Cluster Representation
In Deﬁnition 1, Hawkes processes are introduced by specifying their intensity processes.
One can also deﬁne them using a so-called cluster representation (Jovanovi´c et al., 2015).
This representation lends itself to a straightforward causal interpretation as we will see.
The following describes the cluster representation which can be thought of as a simulation algorithm
and it deﬁnes a Hawkes process given a set of constants { µ α } α ∈ V and a set of nonnegative functions { φ βα } α,β ∈ V .
For each α ∈ V , generation 0 -events are generated from a homogeneous
Poisson process with rate µ α .
Each generation 0-event creates a Hawkes tree , that is, a cluster of events (the clusters are independent).
The events in each cluster are generated recursively.
Each event of type α at time s in a cluster has child events of type β according to an inhomogeneous
Poisson process with rate φ βα ( t − s ) for t > s .
This is illustrated in Figure 2 in which four generation 0-events are shown, each of them starting a cluster.
Parent-child relations are shown using line segments.
Note that φ βα = 0 means that there will be no β -children from an α -event, though there could be β -descendants further down from the α -event, e.g., if there exists γ such that φ γα (cid:54) = 0 and φ βγ (cid:54) = 0 .
The observed Hawkes process is then the superposition of the events in all
Time
C oo r d i na t e p r
o
c
e ss e s clusters.
One should note that when observing a Hawkes process, we only have access to a set of event times and types ( α i , t i ) and not to the cluster structure, that is, the line segments in Figureare unobserved.
The parameters in Φ and R have straightforward interpretations.
From the cluster process deﬁnition of the Hawkes process, it follows that Φ βα is the expected number of child events of type β from an α -event.
More generally, for i = 0 , 1 , 2 , . .
, (Φ i ) βα is the expected number of β -descendants in the i ’th generation of a Hawkes tree rooted at an α -event.
This implies that R βα is the expected total number of β -descendants on such a tree (Jovanovi´c et al., 2015).
2.1. Causal Interpretation
In this section, we will deﬁne what we mean by a causal Hawkes process.
Mogensen (2020) deﬁned a causal Hawkes process by requiring that the set of functions { φ βα } also describes how events enter the intensities in intervened systems where the trajectories of some coordinate processes are determined exogenously.
We will instead use the cluster representation to give a causal interpretation based on a type of injection intervention .
That is, the observational state of the system is given by the Hawkes process as deﬁned above.
An interventional state is deﬁned by a set of deterministic injections , { ( α i , t i ) } i =1 ,...,M , such that each injection is an element in V × R .
An injection is a root in a Hawkes tree and the interventional processes are given as the superposition of all intrinsic and interventional trees.
We then say that a linear Hawkes process is causal if a Hawkes tree rooted at an injection event has the same distribution as an intrinsic, or non-injection, Hawkes tree for all coordinate processes α ∈ V and all trees (interventional and intrinsic) are independent.
In other words, the interventional events propagate in the same way through the system as the intrinsic events.
This causal assumption is seen to correspond closely to the assumption of autonomy or modularity which is often used to deﬁne causal models (Pearl, 2009; Peters et al., 2017).
We note that this type of interventions is a simplistic version of interventions that are used in some ﬁelds of science, e.g., in neuroscience where experimenters may intervene on neurons by means of extracellular electrical stimulation.
This, however, is not expected to break the intrinsic dynamics of the nervous system nor sever causal mechanisms (Mefﬁn et al., 2012; Komarov et al., 2019).
It is also similar to how interventions are often modelled in linear control theory in which they are external forces (possibly dependent on the past of the process) that change the dynamics of the system without necessarily breaking any connections ( ˚Astr¨om and Murray, 2008; Zabczyk, 2020).
Deﬁnition 2 (Causal effects)
We will say that Φ βα is the direct (causal) effect from α on β and that
R βα is the total (causal) effect from α on β .
We will say that φ βα is the (causal) link function from α to β .
The interpretation of the entries in Φ and R and the causal assumption justify the above definitions.
If we inject an interventional event of type α at time s ∈ R , this event on average has Φ βα events of type β as ﬁrst-generation (or direct ) descendants (when considering all t > s ).
The
Hawkes tree rooted at the interventional α -event on average has a total of R βα events of type β , counting the injectional event if α = β .
2.2. Graphical Representation
A graph is a pair ( V, E ) where V is a ﬁnite set of nodes and E is a set of edges, each between a pair of (not necessarily distinct) nodes.
We will use directed mixed graphs (DMGs), i.e., graphs such that every edge is either directed , → , or bidirected , ↔ , and such that between any pair of nodes, α and β , there is a subset of the edges { α → β, α ← β, α ↔ β } .
We use ∼ to denote a generic edge of either type.
If an edge α ∼ β is in a graph G , we write α ∼ G β .
If the edge α → β is in the graph
G , we say that α is a parent of β , and we let pa G ( β ) denote the set of parents of β in the graph G .
Given a causal Hawkes process with coordinate processes indexed by V = { 1 , 2 , . . . , n } , we construct its causal graph , D =
( V, E ) , such that α → D β if and only if φ βα (cid:54) = 0 .
Using the continuity assumption of φ βα , we see that if α → β is in the causal graph, then there exists a nontrivial interval such that φ βα > 0 on this interval.
A walk between α and β is an ordered, alternating sequence of nodes, γ i , and edges, ∼ j ,
[Equation]
such that for each i = 1 , . .
, m the edge ∼ i is between γ i − 1 and γ i .
We say that α and β are endpoint nodes .
A path is a walk such that no node occurs more than once.
We say that a path from α to β is directed if every edge is directed and points towards β , and a directed cycle is a directed path from α to β composed with the edge β → α .
We say that a DMG is acyclic if it contains no directed cycles.
We say that a DMG is a directed graph if it only contains directed edges.
3. Representation as a Linear SEM
The relation between the observed quantity, Σ , and the parameters, (Φ , Λ) , in Equation (2) is seen to be similar to the relation between the parameters and the covariance matrix of a linear structural equation model (linear SEM) and we will leverage this fact.
There is a number of papers with identiﬁcation results in this model class (Brito and Pearl, 2002; Tian, 2007, 2009; Foygel et al., 2012; Chen, 2016; Drton and Weihs, 2016; Weihs et al., 2018), though some results only apply to acyclic linear SEMs.
Bollen (1989) provides an introduction to the model class.
We give a short introduction here to describe the connection to Hawkes processes.
Let X =
( X 1 , . . .
, X n ) T be a vector of random variables and ε = ( ε 1 , . . .
, ε n ) T be a vector of zero-mean noise terms.
Let
X = BX + ε (3) such that the matrix B has zeros on the diagonal.
We let Ω ε denote the covariance of ε .
The matrix
I n − B is invertible if and only if B does not have 1 as an eigenvalue and in this case the covariance of X , denoted by C , is
[Equation]
Often one will assume that some entries of B and Ω ε are zero which can be encoded by a DMG (also known as a mixed graph ) such that if α → β is not in the DMG, then B βα = 0 , and if α ↔ β ( α (cid:54) = β ) is not in the DMG, then (Ω ε ) βα =
(Ω ε ) αβ =
The error terms are often assumed to be
Gaussian.
We note that acyclicity is sometimes assumed, i.e., that the DMG corresponding to the model is acyclic, though we are not making this assumption.
We should note that in the linear SEM literature, authors often parametrize the model in Equation (3) using a transposition of B , X = B T X + ε .
We use this alternative notation as it aligns better with common notation in the Hawkes process literature.
One immediately sees the similarity between Equations (2) and (4) in that the observable lefthand sides of the equations impose similar restrictions on the model parameters, and we use this similarity to translate results to the Hawkes models.
There are two questions that should be addressed.
First, the diagonal of B in the linear SEM is often assumed to be zero, and we will handle this by re-writing Equation (2) slightly (Section 3.2).
Second, Λ is a diagonal matrix in Equation (2) whereas no such restriction on Ω ε is necessary.
We will see that this discrepancy vanishes when we also consider partially observed Hawkes processes (Section 3.1).
3.1. Marginalization
If the Hawkes process is partially observed, only a proper submatrix of Σ in Equation (2) is available, and the coordinate processes, V , are partitioned into a set of observed processes, O , and a set of unobserved processes, U , such that V
= O ˙ ∪ U .
We use a latent projection to represent the graphical structure of the partially observed system.
We say that edges α → β and α ↔ β have a head at β .
Deﬁnition 3 (Latent projection, Verma and Pearl (1990); Richardson et al.
Let G =
( V, E ) be a DMG.
Its latent projection on nodes O is the DMG ¯ G =
( O, F ) such that for α, β ∈
O we have α → β in ¯ G
if there is a directed walk from α to β such that every non-endpoint node is in V \
O .
We have α ↔ β in ¯ G if there is a walk between α and β with heads at both α and β such that every non-endpoint node is a noncollider and is in V \ O .
Note that for a walk in the above deﬁnition the set of non-endpoint nodes can be empty and therefore if α → β is in G , then it is also in its latent projection whenever α, β ∈
O .
We also note that the deﬁnition uses walks to allow for cycles (Mogensen and Hansen, 2020).
In DAG-based models, this is not needed and the deﬁnition is often stated using paths (Richardson et al., 2017).
Using a DMG, G =
( V, E ) , we will describe a parametrization of the observed integrated covariances.
We let M s + ( G ) denote the set of ( n × n ) Schur stable matrices with nonnegative entries such that for α, β ∈ V = { 1 , 2 , . . . , n } and M ∈ M s + ( G ) α
(cid:54)→ G β ⇒ M βα = 0 .
We let M s + , 0 ( G ) ⊆
M s + ( G ) denote the subset of such matrices in which all diagonal elements are zero.
We let M pd + ( G ) denote the set of symmetric, positive deﬁnite matrices with nonnegative entries such that for α, β ∈ V, α (cid:54) =
β , and M ∈ M pd + ( G )
[Equation]
Theorem 4 Let X be a multivariate Hawkes process such that ρ (Φ) < 1 and such that O are the observed coordinate processes and U are the unobserved coordinate processes.
Let D =
( O ˙ ∪ U, E ) be its causal graph.
Then the observed integrated covariance can be written as
[Equation]
such that (Ξ , Θ) ∈
M s + ( G ) × M pd + ( G ) when G is the latent projection of D on O .
Proof Let (Φ , Λ) denote the parameters in the full system, and let Σ denote the full integrated covariance matrix.
Let Σ OO be the observed submatrix of Σ , k
= | O | , and l = | U | .
We have that
I − Ξ is invertible and
[Equation]
where
[Equation]
see Hyttinen et al.
Using Schur complements, we can write (see Appendix A)
[Equation]
Θ is clearly positive semideﬁnite, and positive deﬁnite if Λ is.
Then entries of Ξ and Θ are nonnegative, and Lemma 5 shows that Ξ is Schur stable.
If α is not a parent of β in the latent projection then there is no directed walk from α to β in D such that every nonendpoint node is unobserved and it follows that Ξ βα = 0 using that ( I − A ) − 1 = (cid:80) ∞ i =
A i when ρ ( A ) < 1 .
Similarly, we see that α (cid:54)↔ G β implies that Θ αβ =
Θ βα = 0 for α (cid:54) = β .
Stability under Marginalization
We use the following lemma to show that if the full system is Schur stable, then a partially observed system is also Schur stable.
This result does not hold without the assumption of nonnegative entries in the directed matrix (see Hyttinen et al.
for an example that illustrates this).
Lemma 5 Let A be a square matrix with nonnegative entries such that ρ ( A ) < 1 , and let O ˙ ∪ U be a partition of its rows and columns.
The matrix B =
A OO + A OU ( I − A UU ) − 1 A UO has nonnegative entries and ρ ( B ) < 1 .
Proof The matrix A UU is a principal submatrix of A and therefore ρ ( A UU ) ≤ ρ ( A ) <
(Horn and Johnson, 1985, Corollary 8.1.20).
Therefore ( I − A UU ) − 1 = (cid:80) ∞
i =
( A UU )
i
and it follows that every entry of B is nonnegative.
It holds that I − B is the Schur complement of I − A UU in I − A , that is, I − B
=
( I − A ) /
( I −
A UU ) , and therefore I − B is invertible.
The matrix I − A is a nonsingular M -matrix
so ( I − A ) −has nonnegative entries.
It follows that ( I − B ) − 1 has nonnegative entries and therefore I − B is a nonsingular M-matrix (Berman and Plemmons, 1979, Chapter 6, Theorem 2.3).
Then every real eigenvalue of I − B is positive (Berman and Plemmons, 1979, Chapter 6, Theorem 2.3).
Since B is nonnegative, ρ ( B ) is an eigenvalue of B (Horn and Johnson, 1985, Theorem 8.3.1).
It follows that 1 − ρ ( B ) is an eigenvalue of I − B and therefore 1 − ρ ( B ) > 0 .
3.2. Normalized Effects
When Φ is a matrix of direct effects of a Hawkes process (or marginalized effects, Ξ ), the diagonal elements are not necessarily zero.
On the other hand, in a linear SEM the diagonal of B (Equation (3)) is commonly assumed to be zero.
We introduce the matrix Γ by scaling each row of Ξ by one minus its diagonal element and placing zeros on the diagonal,
[Equation]
= α, and Γ βα = 0 if β = α, and I − Γ is invertible (Hyttinen et al., 2012).
Hyttinen et al.
(2012) study linear SEMs and use the same approach to derive so-called canonical linear cyclic models from models with self-loops.
One should note that Ξ ββ ≤ ρ (Ξ) (Horn and Johnson, 1985, Corollary 8.1.20).
Therefore, if Ξ corresponds to a partially observed linear Hawkes process with ρ (Φ) < 1 , we have ρ (Ξ) <
(Lemma 5) and Ξ ββ (cid:54) = 1 .
We think of the above operation as a type of normalization, and if this is done in the full model (no marginalization), we will say that the normalized parameters, that is, Φ βα / (1 − Φ ββ ) , are normalized direct (causal) effects .
It is seen that a normalized direct effect, Φ βα / (1 − Φ ββ ) , α (cid:54) =
β , is the average number of direct descendants of type β from an α -event, counting every direct selftriggered event in the β -process.
That is, on the Hawkes tree rooted at α , Φ βα / (1 − Φ ββ ) is the expected number of β -events on subtrees of the form α − β − β − . . .
− β for any number of β -events.
Corollary 6 Let X be a multivariate Hawkes process with stationary increments such that O are the observed coordinate processes and U are the unobserved coordinate processes and let D be its causal graph.
Then the observed integrated covariance can be written as
[Equation]
such that (Γ , Ω) ∈ M s 0 , + ( G ) × M pd + ( G ) when G is the latent projection of D on O .
Proof Consider the representation from Theorem 4.
Then we have
[Equation]
Let D denote a diagonal matrix such that the α ’th diagonal element equals 1 − Φ αα and Γ is as deﬁned above.
Then
[Equation]
The claims all follow from Theorem 4.
We should just show that Γ is a Schur stable matrix (Proposition 7).
If we consider a Hawkes process and its causal graph, D =
( V, E ) , we see that the latent projection of D , G =
( O, F ) , O ⊆ V , represents the observed covariance, ¯Σ =
Σ OO in the following way.
If α → β is not in G , then Γ βα = 0 .
For α (cid:54) =
β , if α ↔ β is not in G , then Ω βα =
Ω αβ = 0 .
Omitting self-loops, this corresponds to the graphical representation of a linear
SEM.
This means that the observable integrated covariances after rewriting are parametrized exactly like a linear SEM and we will use this to obtain identiﬁcation results (Section 4) and equality constraints (Section 5).
While the above corollary casts the Hawkes integrated covariance as a set of equations describing the covariance matrix of a linear SEM, the Hawkes parameters are required to be nonnegative.
Proposition 7 Let Ξ be a nonnegative matrix such that ρ (Ξ) < 1 .
Then the normalized matrix, Γ , also satisﬁes ρ (Γ) < 1 .
4. Identiﬁcation
Using constraints on the causal graph, we will discuss the identiﬁcation of the entries of the matrices Ξ and Γ .
To give a formal deﬁnition of identiﬁcation, consider two stationary Hawkes processes,
M 1 ∈ M and M 2 ∈ M , with coordinate processes V 1 and V 2 , respectively, for a model M .
Let ˜ Pand ˜ P 2 be their marginal distributions over a common observed set of coordinate processes, O , let f be a function of such a marginal distribution, and let α, β ∈
O .
Finally, let Π βα be a parameter ( Π βα = Ξ βα or Π βα = Γ βα ).
We will say that the parameter Π βα is identiﬁed from f ( ˜ P ) in the model M if f ( ˜ P 1 ) =
f
( ˜ P 2 ) implies Π 1 βα =
Π 2 βα where Π iβα is the parameter value for M i .
Note that while identiﬁcation is sometimes only discussed from the observed distribution in its entirety, we use f to distinguish simpler statistics that can be sufﬁcient for identiﬁcation.
As we will see, under graphical constraints some parameters are identiﬁed from the integrated covariance matrix in which case the inferential problem is reduced to a ﬁnite-dimensional one.
4.1. Connection to Linear SEMs
There are several papers describing graphical conditions to ensure identiﬁability of parameters of a linear structural equation model.
For these results, one assumes a certain graphical structure represented by a DMG which encodes sparsity in the matrices B and Ω ε in Equation (4).
Identiﬁcation of a parameter means that for a ﬁxed graphical structure and an observed covariance, C , this parameter is uniquely determined.
The idea is now that one can use any available identiﬁcation result from the (cyclic) linear structural equation toolbox to provide identiﬁcation results for the entries in Γ such as those provided by, e.g., Foygel et al.
; Chen (2016);
Weihs et al.
This follows
5 A
B
C from noting that if we use the latent projection of the Hawkes causal graph to encode sparsity, the problem in Corollary 6 is the same as the linear SEM problem, though with a smaller parameter space as Γ and Ω are required to be nonnegative and Γ is required to be Schur stable.
While any identiﬁcation result from the linear SEM problem therefore applies to the problem in Corollary 6, one can actually obtain stronger results using that the parameter space is smaller.
Section 4.2 shows this by an example.
We also note that one typically has to distinguish between global and generic identiﬁcation of parameters.
Example 8 Consider the graph in Figure 3 A and assume that this is the causal graph of a Hawkes process on nodes V = { 1 , 2 , 3 , 4 , 5 } such that coordinate process 5 is unobserved.
Let Γ be the 4 × 4 matrix of direct effects.
If we apply the identiﬁcation criterion of Weihs et al.
(2018), we ﬁnd that every Γ βα such that α, β ∈
O is identiﬁed from Σ (note that Γ βα =
Φ βα / (1 − Φ ββ ) for α, β ∈
O , α (cid:54) =
β , in this example as Φ UO = 0 ).
It follows that Φ 11 , Φ 32 , and Φ 33 are identiﬁed from the integrated mean and covariance.
On the other hand, one can show by direct computation that Φ 21 , Φ 22 , Φ 24 , Φ 43 , and Φ 44 are not identiﬁed.
If we consider Φ 21 , we can understand the lack of identication from the fact that Φcannot be identiﬁed.
Intuitively, this is explained by its unobserved parent process which implies that we cannot, using only the integrated covariance, separate the contributions from coordinate processes 2 and 5 to the observed integrated covariance.
4.2. Identiﬁcation of Cycles
The n -cycle, D n , is the directed graph on nodes α 1 , . .
, α n such that α i → α i +1 for all i = 1 , . . .
, n − 1 and α n → α 1 .
When n ≥ 3 , Drton et al.
(2011) show that Equation (4) has at most two solutions when the graph is an n -cycle and that generically it has two solutions.
We show in the following that at most one of them is in M s + , 0 ( G )
× M pd + ( G ) , thus, is a solution to the normalized
Hawkes problem.
Proposition 9 Let G be an n -cycle, n ≥ 3 .
There is at most one solution to
[Equation]
such that (Γ , Ω) ∈ M s 0 , + ( G ) × M pd + ( G ) .
5. Equality Constraints
It is well-known that partially observed DAG-models satisfy constraints that are not described by conditional independence some of which are known as equality constraints .
Richardson et al.
(2017) treat the general (acyclic) case while Chen et al.
; Chen (2016) consider equality constraints in linear SEMs.
In DAG-based models, equality constraints are testable implications that may bring strictly more information about the underlying structure than using conditional independence alone.
We will see that the analogous statement holds for the Hawkes process models, though the relevant notion of independence to compare with is that of local independence .
Local independence is an asymmetric notion of independence of the coordinate processes of a multivariate stochastic process (Schweder, 1970; Aalen, 1987; Didelez, 2000, 2008).
The following deﬁnition is speciﬁc to point processes, however, it can be extended to other classes of stochastic processes (Aalen, 1987;
Didelez, 2006; Mogensen et al., 2018; Mogensen and Hansen, 2020).
Deﬁnition 10 (Local independence) Let X
= ( X 1 , . . .
, X n ) T be a multivariate point process with intensity processes λ and let V = { 1 , . . . , n } .
Let A, B, C, D ⊆ V .
Let F Dt denote the natural ﬁltration of X D , i.e., the ﬁltration generated by the coordinate processes in D .
We say that X B is locally independent of X A given X C if for all β ∈ B and for all t ∈ R ,
[Equation]
[Equation]
is adapted to the ﬁltration F Ct .
5.1. Structure learning in Hawkes processes
A linear Hawkes process has a very simple ‘mechanistic’ interpretation which is evident from both the intensity representation in Deﬁnition 1 and the cluster representation (e.g., Figure 2).
In the intensity process of β , a function φ βα concisely describes how an α -event increases the intensity with which future β -events occur.
This makes Hawkes processes attractive in settings where an analyst is not merely trying to ﬁt a process to an observed distribution but is also interested in understanding how events in some coordinate processes create events in other coordinate processes.
Examples include social media networks, systems of interacting neurons, and alarm networks in complex engineered systems.
In many applications, the observed data only include events in a proper subset of the coordinate processes of the full system.
Local independence has been used for structure learning in stochastic process models (Meek, 2014; Mogensen et al., 2018;
Thams, 2019; Mogensen, 2020), analogously to how conditional independence is used for constraint-based structure learning in classical models (Spirtes et al., 2000;
Spirtes and Zhang, 2019).
Structure learning based on local independence typically assumes faithfulness , that is, that X B is locally independent of X A given X C if and only if B is µ -separated from A given C in G , the DMG representing the model ( µ -separation, or δ -separation, is a concept analogous to m -, or d -separation (Didelez, 2000, 2008; Mogensen and Hansen, 2020)).
The Markov equivalence class of G is the set of DMGs that encode exactly the same set of µ -separations, hence local independencies.
Mogensen and Hansen (2020) show that each Markov equivalence class contains a unique maximal element and this graph is therefore a natural learning target when using tests of local independence.
An obvious question is therefore whether there are constraints imposed by the structure that are not described by local independence, or µ -separation.
This is indeed the case and such constraints can be used to distinguish between graphical structures that imply the same set of local independencies.
In the next subsection, we show that one can use the integrated mean and covariance to ﬁnd such constraints.
In applications, this means that the set of graphical structures that can explain the observed data may be reduced, thus we may obtain strictly more information about how coordinate processes interact.
5.2. Constraints on the Integrated Covariance
From Equation (7) and the section on marginalization we see that every equality constraint in a linear structural equation model implied by the graphical structure imposes the same constraint on the parameters (Γ , Ω) and this allows us to ﬁnd constraints in the Hawkes model.
Chen et al.
(2014) describe how to ﬁnd equality constraints in linear SEMs, more speciﬁcally through overidentiﬁcation of model parameters.
When parameters in the linear SEM are identiﬁed they can be computed from the observed covariance matrix.
Overidentiﬁcation occurs when there exist two ways to identify a parameter through different functions of the covariance matrix and based on logically independent model assumptions.
An equality constraint arises from equating such functions.
Chen et al.
(2014) provide algorithms for systematically ﬁnding different identifying functions based on different parts of the model structure.
We will show by an example that equality constraints may provide information about the underlying structure of a linear Hawkes process which is not contained in its ( µ -separation)
Markov equivalence class.
Example 11 Consider again the example illustrated by the graph in Figure 3 B .
If we think of the graph as representing a linear SEM, we can, e.g., use Theorem 1 and Lemma 1 of Chen et al.
(2014) to see that the graphical structure imposes the constraint
[Equation]
and therefore this algebraic constraint is also satisﬁed by the Hawkes model corresponding to the graph (note that Γ 32 is identiﬁed and that the constraint corresponds to two ways of identifying Γ 43 ).
The graphs B and C in Figure 3 (right) imply the same set of local independencies when using µ -separation.
However, the above constraint is satisﬁed in B only and therefore this constraint allows us to discriminate between the two models.
In DAG-based models, equality constraints may be thought of as conditional independence constraints in identiﬁed interventional distributions (Richardson et al., 2017).
This is also seen to be the case in the linear Hawkes setting.
First, we deﬁne a type of intervention different from the injection intervention in Section 2.1.
Assume we have a linear Hawkes process as deﬁned by a set of link functions { φ βα } α,β ∈ V and a set of constants { µ α } α ∈ V with (normalized) parameters (Γ , Ω) .
In this intervention and for a subset of coordinate processes I ⊂ V , we set ¯ φ βα = 0 and ¯ µ β = 1 whenever β ∈ I and otherwise ¯ φ βα = φ βα and ¯ µ β = µ β .
The intervened process is the linear Hawkes process deﬁned by { ¯ φ βα } α,β ∈ V and { ¯ µ α } α ∈ V .
Note that ¯Φ is Schur stable.
This intervention implies that all events in the processes I are exogenous (generation 0 in the cluster representation).
Consider also the corresponding linear SEM with variables indexed by I such that B = Γ and Ω ε
= Ω in Equation (4).
The observed integrated Hawkes covariance in the interventional process is then equal to the interventional covariance in this linear SEM when variables in I are made exogenous with zero mean and unit variance.
This means that when equality constraints can be thought of as constraints in an interventional distribution in the linear SEM, they can also be thought of as constraints on the observed integrated Hawkes covariance of an interventional process.
6. Conclusion
We obtained identiﬁcation results and equality constraints for Hawkes process models through the similarity between the integrated Hawkes covariance and the observed covariance of a linear structural equation model.
These equality constraints are useful for structure learning as they can supplement constraint-based learning based on tests of local independence.
The approach in this paper ﬁnds equality constraints through a time-independent statistic which may be applicable in other classes of stochastic processes as well.
A Hawkes process is deﬁned using the set of φ βα -functions along with the µ α -constants for α, β in the ﬁnite set V and a Hawkes process model is in that sense nonparametric.
Constraints that arise from the integrated covariance offer also a dimension reduction as these constraints exist in a ﬁnite-dimensional parameter space instead of a function space.
This makes them more suitable to employ in data analysis as estimation of the link functions, { φ βα } , is a challenging problem even in the case of full observation and partial observation only adds to the complexity of this task.
Acknowledgments
This work was supported by a research grant from VILLUM FONDEN (13358) and by a DFFInternational Postdoctoral Grant (0164-00023B) from Independent Research Fund Denmark.
The author thanks Niels Richard Hansen, Ilya Shpitser, and Daniel Malinsky for helpful discussions.
Appendix A. Marginalization
In this section and for a matrix A , we deﬁne A − 1¯
I ¯ J =
( A − 1 ) ¯ I ¯ J and A − T ¯
I ¯ J
=
( A − T )
¯ I ¯ J .
Identity matrices of different dimensions are all denoted by I .
Consider an invertible block matrix
[Equation]
such that D is invertible.
We let M/D
=
A − BD − 1 C denote the Schur complement of the block matrix D in the matrix M .
It holds that
[Equation]
Matrices I − Φ
and I − Φ UU are both invertible as I − Φ is a nonsingular M-matrix (Chapter 6,
Theorem 2.3 in Berman and Plemmons (1979)).
Therefore,
[Equation]
by deﬁnition of Ξ .
Furthermore,
[Equation]
Let I = I n and A ¯ I
=
A ¯ I ¯ I .
From the above, it follows that
[Equation]
Appendix B. Proofs
Proof [Proposition 7] Assume to obtain a contradiction that ρ (Γ) ≥ 1 and let λ = ρ (Γ) .
The matrix Γ is nonnegative and therefore λ is an eigenvalue of Γ and a nonnegative eigenvector, x (cid:54)
= 0 , can be chosen (Horn and Johnson, 1985, Theorem 8.3.1), Γ x = λx .
Let D Ξ denote the diagonal matrix such that ( D Ξ ) ii =
Ξ ii for all i .
From the deﬁnition of Γ , we have
[Equation]
Multiplying with x and using Γ x = λx , we obtain
[Equation]
For all i , ( D Ξ ) ii =
Ξ ii ≤ ρ (Ξ) < 1 (Horn and Johnson, 1985, Corollary 8.1.20).
We have that 1 − λ ≤ 0 and that the entries of x are nonnegative and therefore
[Equation]
where the inequality should be read entrywise.
It holds that x (cid:54) = 0 and therefore Ξ x ≥ x implies that ρ (Ξ) ≥ 1 which is a contradiction (Horn and Johnson, 1985, Theorem 8.3.2).
Proof [Proposition 9] Let (Γ 0 , Ω 0 ) , (Γ 1 , Ω 1 ) be the two solutions of a linear SEM corresponding to an n -cycle (generically, they are distinct).
Let δ ji denote the inverse of the i ’th diagonal element of Ω j and let λ ji denote the ( i, i + 1) -entry of Γ j (modulo n ).
Drton et al.
(2011) show that if λ 0 i =for some i then there is at most one solution.
Assuming that λ 0 i (cid:54)
=
for all i = 1 , . . .
, n , they show that
[Equation]
where K is the inverse of the observed covariance, Σ , and K − i denotes the submatrix of K with the i ’th row and column deleted.
Replacing the ﬁrst δ 1 i in the lower equation with the expression in the upper equation we obtain,
[Equation]
We assume that the observed covariance is from a Schur stable system, say ρ (Γ 0 ) <
Then every eigenvalue of Γ 0 has absolute value less than 1, and | det(Γ 0 )
| < 1 .
Due to the structure of Γ 0 , | det(Γ 0 ) | = (cid:81) i λ
i such that (cid:81) i λ
i < 1 and (cid:81) i ( λ 0 i ) 2 < 1 .
Every δ 0 i is positive, and this means that the right-hand side of Equation (8) is negative.
Every δ 1 i and every λ
i are positive and therefore we must have ( (cid:81) i λ 1 i ) 2 − 1 > 0 and (cid:81) i λ
1 i > 1 .
This means that | det(Γ 1 ) | > 1 and therefore ρ (Γ 1 ) ≥ 1 .