Original: Tractable Latent State Inference for Hidden Continuous-Time semi-Markov Chains
Revised: Tractable Latent State Inference for Hidden Continuous-Time semi-Markov Chains

Original: Abstract
Revised: Abstract

Original: Hidden semi-Markov Models (HSMM’s) - while broadly in use - are restricted toa discrete and uniform time grid.
Revised: Hidden semi-Markov Models (HSMM’s) - while broadly in use - are restricted toa discrete and uniform time grid.

Original: They are thus not well suited to explain oftenirregularly spaced discrete event data from continuous-time phenomena.
Revised: They are thus not well suited to explain oftenirregularly spaced discrete event data from continuous-time phenomena.

Original: We showthat non-sampling-based latent state inference used in HSMM’s can be generalizedto latent Continuous-Time semi-Markov Chains (CTSMC’s).
Revised: We showthat non-sampling-based latent state inference used in HSMM’s can be generalizedto latent Continuous-Time semi-Markov Chains (CTSMC’s).

Original: We formulate integrodifferential forward and backward equations adjusted to the observation likelihoodand introduce an exact integral equation for the Bayesian posterior marginals and ascalable Viterbi-type algorithm for posterior path estimates.
Revised: We formulate integrodifferential forward and backward equations adjusted to the observation likelihoodand introduce an exact integral equation for the Bayesian posterior marginals and ascalable Viterbi-type algorithm for posterior path estimates.

Original: All presented equationscan be efﬁciently solved using existing numerical methods.
Revised: All presented equationscan be efﬁciently solved using existing numerical methods.

Original: We evaluate ourapproaches in latent state inference scenarios in comparison to classical HSMM’s.
Revised: We evaluate ourapproaches in latent state inference scenarios in comparison to classical HSMM’s.

Original:  Introduction
Revised:  Introduction

Original: Continuous-Time semi-Markov Chains (CTSMC’s) comprise the continuous time analogue of(discrete-time) semi-Markov chains.
Revised: Continuous-Time semi-Markov Chains (CTSMC’s) comprise the continuous time analogue of(discrete-time) semi-Markov chains.

Original: In comparison to Markov chains, semi-Markov chains allow to model non-exponential, arbitrary waiting times between state changes.
Revised: In comparison to Markov chains, semi-Markov chains allow to model non-exponential, arbitrary waiting times between state changes.

Original: This more accuratelymatches such times occurring in many scientiﬁc ﬁelds [11].
Revised: This more accuratelymatches such times occurring in many scientiﬁc ﬁelds [11].

Original: Hidden semi-Markov Models (HSMM’s)on ﬁnite state spaces have seen wide application and thorough research [22].
Revised: Hidden semi-Markov Models (HSMM’s)on ﬁnite state spaces have seen wide application and thorough research [22].

Original: However, confrontedwith measurements from continuous-time systems, their adaptivity is limited and computation mightbecome unnecessarily expensive.
Revised: However, confrontedwith measurements from continuous-time systems, their adaptivity is limited and computation mightbecome unnecessarily expensive.

Original: Because of that, continuous-time analogues of HSMM’s have seenincreased research interest lately [17, 1, 6, 7].
Revised: Because of that, continuous time analogues of HSMM’s have seenincreased research interest lately [17, 1, 6, 7].

Original: However, latent state inference is hard in continuoustime, since the usual methods demand a reformulation of the process to obtain the Markov property.
Revised: However, latent state inference is hard in continuoustime, since the usual methods demand a reformulation of the process to obtain the Markov property.

Original: Markovianization by state space expansion as used for HSMM’s demands propagation of asemi-inﬁnite uncountable state for latent CTSMC’s
Revised: Markovianization by state space expansion as used for HSMM’s demands propagation of asemi-inﬁnite uncountable state for latent CTSMC’s

Original: even if the original state space is ﬁnite.
Revised: even if the original state space is ﬁnite.

Original: Thus,besides ad-hoc discretization, two classes of solution methods have emerged.
Revised: Thus,besides ad-hoc discretization, two classes of solution methods have emerged.

Original: First, a variety ofsampling-based solutions has been published, often targeted at even generalized problems [7, 2, 1, 9].
Revised: First, a variety ofsampling-based solutions has been published, often targeted at even generalized problems [7, 2, 1, 9].

Original: Second, if non-sampling based solutions are sought, state-space expansions based on approximationsof waiting time measures are available [16].
Revised: Second, if non-sampling based solutions are sought, state-space expansions based on approximationsof waiting time measures are available [16].

Original: This second class, by detour, essentially tries to approximate Kolmogorov’s (forward/backward) equations.
Revised: This second class, by detour, essentially tries to approximate Kolmogorov’s (forward/backward) equations.

Original: However, a general framework - not tailoredto speciﬁc families of waiting time measures - is still lacking.
Revised: However, a general framework - not tailoredto speciﬁc families of waiting time measures - is still lacking.

Original: Kolmogorov’s equations in integrodifferential form have been studied recently [20].
Revised: Kolmogorov’s equations in integrodifferential form have been studied recently [20].

Original: Although initial or terminal states are given, theequations rely on speciﬁc boundary conditions to restrict the non-Markovian process to the consideredtime-window.
Revised: Although initial or terminal states are given, theequations rely on speciﬁc boundary conditions to restrict the non-Markovian process to the consideredtime-window.

Original: Fixed conditions hinder the equations’ usage in latent state inference which typicallyspans many such time-windows.
Revised: Fixed conditions hinder the equations’ usage in latent state inference which typicallyspans many such time-windows.

Original: To alleviate this restriction, we show how to adjust the memoryterms in Kolmogorov’s equations to include observations.
Revised: To alleviate this restriction, we show how to adjust the memoryterms in Kolmogorov’s equations to include observations.

Original: As a consequence, we provide generalforward and a backward algorithms, each encoded in a single inhomogeneous integro-differentialequation.
Revised: As a consequence, we provide generalforward and a backward algorithms, each encoded in a single inhomogeneous integro-differentialequation.

Original: We also provide a method for exact Bayesian smoothing which uses the quantities obtainedfrom the forward and backward algorithms.
Revised: We also provide a method for exact Bayesian smoothing which uses the quantities obtainedfrom the forward and backward algorithms.

Original: As a corollary, we propose a Viterbi-type approach formaximum-a-posteriori path estimation and sketch a method to implement adaptive step-size HSMM’s that keep their homogenous properties.
Revised: As a corollary, we propose a Viterbi-type approach formaximum-a-posteriori path estimation and sketch a method to implement adaptive step-size HSMM’s that keep their homogenous properties.

Original: The resulting general framework may incorporate arbitrarynumerical solvers  or make use of approximations by the mentioned second class of solution methods.
Revised: The resulting general framework may incorporate arbitrarynumerical solvers available or make use of approximations by the mentioned second class of solutionmethods.

Original: Related Work.
Revised: Related Work.

Original: Continuous-time HMM’s and their inference has recently been compiled by [14]).
Revised: Continuous-time HMM’s and their inference has recently been compiled by [14]).

Original: Although tailored to a speciﬁc problem, an interesting MAP path estimation method for CT-HSMM’sbased on stochastic state classes has been published in [5].
Revised: Although tailored to a speciﬁc problem, an interesting MAP path estimation method for CT-HSMM’sbased on stochastic state classes has been published in [5].

Original: Neural ODE’s for the progression ofmemory variables have been investigated in [7]).
Revised: Neural ODE’s for the progression ofmemory variables have been investigated in [7]).

Original: An application of CT-HSMM’s including parameterinference using a discretized waiting time variable [6, 17].
Revised: An application of CT-HSMM’s including parameterinference using a discretized waiting time variable [6, 17].

Original: Generalized HSMM’s that are losslesscontinuous discretizations but the transitions are observable has been proposed in [21].
Revised: Generalized HSMM’s that are losslesscontinuous discretizations but the transitions are observable has been proposed in [21].

Original: An elaborateforward-ﬁltering-backward-sampling for a HASMM [1].
Revised: An elaborateforward-ﬁltering-backward-sampling for a HASMM [1].

Original: To our knowledge, we are the ﬁrst tointroduce a "traditional" form of a forward-backward algorithm for a latent CTSMC.
Revised: To our knowledge, we are the ﬁrst tointroduce a "traditional" form of a forward-backward algorithm for a latent CTSMC.

Original: Also, while [ 5 ]is able to do MAP estimates of trajectories as well, their method is very different to our approach.
Revised: Also, while [ 5 ]is able to do MAP estimates of trajectories as well, their method is very different to our approach.

Original: Notation.
Revised: Notation.

Original: To maintain readability, we  use the symbol P to denote density and mass functions alike.
Revised: To maintain readability, we will use the symbol P to denote density and mass functionsalike.

Original: Which kind is needed should be clear from the context.
Revised: Which kind is needed should be clear from the context.

Original: Nonetheless, we refer to appendix section Afor derivations and proofs using a stricter notation style.
Revised: Nonetheless, we refer to appendixsection A for derivations and proofs using a stricter notation style.

Original: 1.1 Continuous-Time semi-Markov Chains
Revised: 1.1 Continuous-Time semi-Markov Chains

Original: A continuous-time semi-Markov chain (CTSMC) { X ( t ) } t ≥ 0 on a countable state space X is astochastic jump process.
Revised: A continuous-time semi-Markov chain (CTSMC) { X ( t ) } t ≥ 0 on a countable state space X is astochastic jump process.

Original: The general process is described by transition rates λ : X × R ≥ 

Revised: The general process is described by transition rates λ : X × R ≥

Original: × R ≥ 0 ×
Revised: × R ≥ 0 ×

Original: X → R ≥ 0 .
Revised: X → R ≥ 0 .

Original: Conditioned on the current state x ∈ X at time t ∈ R ≥ 0 and the time τ since the lasttransition at t − τ < t , the rates λ ( x, τ, t ; x (cid:48) ) describe the probability per unit time of transitioningshortly after time t to state x (cid:48) .
Revised: Conditioned on the current state x ∈ X at time t ∈ R ≥ 0 and the time τ since the lasttransition at t − τ < t , the rates λ ( x, τ, t ; x (cid:48) ) describe the probability per unit time of transitioningshortly after time t to state x (cid:48) .

Original: These transition rates sufﬁce to resolve the waiting times and jumpdirections for each tuple ( x, τ ) at all times t .
Revised: These transition rates sufﬁce to resolve the waiting times and jumpdirections for each tuple ( x, τ ) at all times t .

Original: In this work, we restrict ourselves to CTSMC’s which arehomogeneous and direction-time independent.
Revised: In this work, we restrict ourselves to CTSMC’s which arehomogeneous and direction-time independent.

Original: The former means that λ ( x, τ, t ; x (cid:48) )
Revised: The former means that λ ( x, τ, t ; x (cid:48) )

Original: ≡ λ ( x, τ ; x (cid:48) )
Revised: ≡ λ ( x, τ ; x (cid:48) )

Original: at all times t .
Revised: at all times t .

Original: The latter additionally implies the factorization λ ( x, τ ; x (cid:48) )
Revised: The latter additionally implies the factorization λ ( x, τ ; x (cid:48) )

Original: ≡ m ( x (cid:48) | x ) λ ( x, τ )with the exit rate λ ( x, τ ) ≡ (cid:80) x (cid:48) (cid:54) =
Revised: ≡ m ( x (cid:48) | x ) λ ( x, τ )with the exit rate λ ( x, τ ) ≡ (cid:80) x (cid:48) (cid:54) =

Original: x λ ( x, τ ; x (cid:48) ) and the transition probabilities of the embedded
Revised: x λ ( x, τ ; x (cid:48) ) and the transition probabilities of the embedded

Original: Markov chain m ( x (cid:48) | x ) .
Revised: Markov chain m ( x (cid:48) | x ) .

Original: Satisfying (cid:80) x (cid:48) (cid:54) =
Revised: Satisfying (cid:80) x (cid:48) (cid:54) =

Original: x m ( x (cid:48) | x ) = 1 , i.e., excluding self-transitions, theyform a categorical distribution over next states x (cid:48) (cid:54)
Revised: x m ( x (cid:48) | x ) = 1 , i.e., excluding self-transitions, theyform a categorical distribution over next states x (cid:48) (cid:54)

Original: =
Revised: =

Original: x .
Revised: x .

Original: Under direction-time independence, thewaiting time in each state is thus statistically independent of the jump direction.
Revised: Under direction-time independence, thewaiting time in each state is thus statistically independent of the jump direction.

Original: In this setting, for each state x , we have the waiting time measures F ( τ | x ) = 1 − Λ ( τ | x )with survival/tail functions Λ ( τ | x ) =
Revised: In this setting, for each state x , we have the waiting time measures F ( τ | x ) = 1 − Λ ( τ | x )with survival/tail functions Λ ( τ | x ) =

Original: exp (cid:0) − (cid:82) τ 0 d τ (cid:48) λ
Revised: exp (cid:0) − (cid:82) τ 0 d τ (cid:48) λ

Original: ( x, τ (cid:48) ) (cid:1) and their densities f
Revised: ( x, τ (cid:48) ) (cid:1) and their densities f

Original: ( τ | x ) =
Revised: ( τ | x ) =

Original: λ ( x, τ ) Λ ( τ | x ) .
Revised: λ ( x, τ ) Λ ( τ | x ) .

Original: For convenience of notation, we also introduce the forward transition operator ofthe embedded Markov chain M acting on a test-function g by M g ( x, t ) ≡ (cid:80) x (cid:48)
Revised: For convenience of notation, we also introduce the forward transition operator ofthe embedded Markov chain M acting on a test-function g by M g ( x, t ) ≡ (cid:80) x (cid:48)

Original: (cid:54) =
Revised: (cid:54) =

Original: x m ( x | x (cid:48) ) g ( x (cid:48) , t )and its backward adjoint M † , for which M † g ( x, t ) ≡ (cid:80) x (cid:48) (cid:54) =
Revised: x m ( x | x (cid:48) ) g ( x (cid:48) , t )and its backward adjoint M † , for which M † g ( x, t ) ≡ (cid:80) x (cid:48) (cid:54) =

Original: x m ( x (cid:48) | x ) g ( x (cid:48) , t ) holds.
Revised: x m ( x (cid:48) | x ) g ( x (cid:48) , t ) holds.

Original: Further, a homogeneous continuous-time Markov chain (CTMC) is a homogeneous and direction-timeindependent CTSMC which has constant rates in each state.
Revised: Further, a homogeneous continuous-time Markov chain (CTMC) is a homogeneous and direction-timeindependent CTSMC which has constant rates in each state.

Original: As such, the rates λ ( x, τ ) ≡ λ ( x ) areleft dependent only on the current state.
Revised: As such, the rates λ ( x, τ ) ≡ λ ( x ) areleft dependent only on the current state.

Original: Thus, the waiting time measures are strictly exponentialand so F ( τ | x ) = 1 − exp ( − λ ( x ) τ ) and f ( τ | x ) =
Revised: Thus, the waiting time measures are strictly exponentialand so F ( τ | x ) = 1 − exp ( − λ ( x ) τ ) and f ( τ | x ) =

Original: λ ( x ) exp ( − λ ( x ) τ ) .
Revised: λ ( x ) exp ( − λ ( x ) τ ) .

Original: Though signiﬁcantlyless expressive, inference in CTMC’s is by large parts available analytically [18].
Revised: Though signiﬁcantlyless expressive, inference in CTMC’s is by large parts available analytically [18].

Original: 1.2 Observation Model
Revised: 1.2 Observation Model

Original: In this work, we consider a hidden continuous-time semi-Markov model, where the latent processis a CTSMC { X ( t ) } t ≥ 0 .
Revised: In this work, we consider a hidden continuous-time semi-Markov model, where the latent process is a CTSMC { X ( t ) } t ≥ 0 .

Original: Its state X ( t ) is observable indirectly by means of a dependent randomvariable Y ( t ) taking values in a measurable space ( Y , A ) .
Revised: Its state X ( t ) is observable indirectly by means of a dependent random variable Y ( t ) ∈

Original: We assume to collect a set of observations y 0 , y 1 , . . . y K ∈ A only at discrete time instances t 0 < t 1 < · · · < t  K .
Revised: Y on an observation space Y available only at discrete time instances t 0 < t 1 < · · · < t Kassuming values y 0 , y 1 , . . , y K .

Original: The conditional P ( Y ( t k ) ∈
Revised: The conditional Pr( Y ( t k ) ≤ y k

Original: y k | X ( t k ) = x ) is assumed to be known at observation instances.
Revised: | X ( t k ) = x ) is assumedto be known at observation instances.

Revised: We collect all discrete event data in the measurement set

Revised: E ≡ { ( y k , t k ) | k ∈ { 0 , 1 , . . .

Original:  Finally, wewill often write events in the form y [ t (cid:48) ,t ) denoting the joint event obtained by intersecting singleobservation events, i.e., y [ t (cid:48) ,t ) ≡ (cid:84) t k ∈ [ t (cid:48) ,t ){ Y ( t k )∈
Revised: , K }} . We also deﬁne the parametrizable subset E ( t (cid:48) , t ) ⊆ E whichextracts all samples within a speciﬁc time window E ( t (cid:48) , t ) ≡ { ( y k , t k ) | ( y k , t k ) ∈ E , t k ∈ [ t (cid:48) , t ) } . Finally, we will often write events in the form y [ t (cid:48) ,t ) denoting the joint event obtained by intersectingsingle observation events, i.e., y [ t (cid:48) ,t ) ≡ (cid:110)(cid:84) ( y k ,t k ) ∈E ( t (cid:48) ,t ) Y ( t k )

Original: y k } .
Revised: [Equation]

Original: 1.3 General Idea
Revised: 1.3 General Idea

Original: Latent state inference in Markov processes generally relies on the exploitation of their Markovproperty.
Revised: Latent state inference in Markov processes generally relies on the exploitation of their Markovproperty.

Original: It allows to describe the posterior latent state probability as a product of a forward andbackward term.
Revised: It allows to describe the posterior latent state probability as a product of a forward andbackward term.

Original: While the forward terms account for the probability of past observations at any timeinstance t , the backward terms describe the likelihood of future observations starting from t .
Revised: While the forward terms account for the probability of past observations at any timeinstance t , the backward terms describe the likelihood of future observations starting from t .

Original: In the semi-Markov case, such a factorization is generally not possible, i.e. for a state X ( t ) =
Revised: In thesemi-Markov case, such a factorization is generally not possible, i.e. for a state X ( t ) =

Original: x t
Revised: x t

Original: [Equation] (cid:124) (cid:123)(cid:122) (cid:125)
Revised: P

Original: [Equation]
Revised: [Equation]

Original: (cid:1) (cid:124) (cid:123)(cid:122) (cid:125) .

Original: posterior (cid:54)∝ forward × backward (1)
Revised: posterior (cid:54)∝ forward × backward (1)

Original: However, as the name suggests, semi-Markov processes do not always violate the Markov property.
Revised: However, as the name suggests, semi-Markov processes do not always violate the Markov property.

Original: In particular, the process is Markov at jump instances [13].
Revised: In particular, the process is Markov at jump instances.

Original: Thus, with respect to state changes, wecan again ﬁnd a factorization similar to (1).
Revised: Thus, with respect to state changes, we canagain ﬁnd a factorization similar to (1).

Original: To be made precise later, let ∆ t denote the event of X ( t )transitioning to a state x ∈ X from a different state X ( t − ) (cid:54) =
Revised: To be made precise later, let ∆ t denote the event of X ( t )transitioning to a state x ∈ X from a different state X ( t − ) (cid:54) =

Original: x at time instance t .
Revised: x at time instance t .

Original: Then, in contrastto (1) it holds that
Revised: Then, in contrastto (1) it holds that

Original: [Equation]
Revised: [Equation]

Original:  posterior ∝ forward × backward (2) where P now takes on the form of a density instead of the mass functions in (1).
Revised: (cid:124) (cid:123)(cid:122) (cid:125) , posterior ∝ forward × backward (2) where P now takes on the form of a density instead of the mass functions in (1).

Original: The term P (∆ t |y [0 ,t ) ) in (2) can be calculated under history dependence while the term P
Revised: The term P (∆ t |y [0 ,t ) ) in (2) can be calculated under history dependence while the term P

Original: ( y [ t,T ) | ∆ t ) is bydeﬁnition independent of any past trajectory up to t since it is conditioned on the state change att .
Revised: ( y [ t,T ) | ∆ t ) is bydeﬁnition independent of any past trajectory up to t since it is conditioned on the state change att .

Original: In the following, we refer to such quantities like those occuring on the right-hand side of (2) as"currents".
Revised: In the following, we refer to such quantities like those occuring on the right-hand side of (2) as"currents".

Original:  Forward and Backward Currents
Revised:  Forward and Backward Currents

Original: Probability currents - in the context of Markov chains - are known from the balance equation [19].
Revised: Probability currents - in the context of Markov chains - are known from the balance equation [19].

Original: They also occur in the description of other stochastic processes [12], CTSMC’s in particular aswell [15, 3].
Revised: They also occur in the description of other stochastic processes [12], CTSMC’s in particular aswell [15, 3].

Original: In this section, we establish a general framework of input and output probability andlikelihood currents under observations which is needed for the inference methods afterwards.
Revised: In this section, we establish a general framework of input and output probability andlikelihood currents under observations which is needed for the inference methods afterwards.

Original: 2.1 Input and Output Currents in Continuous-Time semi-Markov Chains
Revised: 2.1 Input and Output Currents in Continuous-Time semi-Markov Chains

Original: Consider again a CTSMC { X ( t ) } t ≥ 0 on a countable state space X .
Revised: Consider again a CTSMC { X ( t ) } t ≥ 0 on a countable state space X .

Original: Let a state x ∈ X of the chainand a time instance t ∈ R ≥ 0 be given.
Revised: Let a state x ∈ X of the chainand a time instance t ∈ R ≥ 0 be given.

Original: The following events form the core of the following section.
Revised: The following events form the core of the following section.

Original: Let ∇ h ( x, t ) ≡ { X ( t ) 
Revised: Let ∆ h ( x, t ) ≡ { X ( t ) (cid:54) =

Original: = x, X ( t + h )(cid:54) = x } denote the joint event of being in two different statesx and some x (cid:48) (cid:54) =
Revised: x, X ( t + h ) = x } denote the joint event of being in two different statesx and some x (cid:48) (cid:54) =

Original: x at the beginning and end of a time-window of length h past t , i.e. the event ofleaving state x between t and t + h .
Revised: x at the beginning and end of a time-window of length h past t , i.e. the event ofleaving state x between t and t + h .

Original: Let in contrast further ∆ h ( x, t ) ≡ { X ( t )(cid:54) =
Revised: Let in contrast further ∇ h ( x, t ) ≡ { X ( t ) =

Original: x, X ( t + h )  = x }denote the joint event of being in state x (cid:48) (cid:54) =
Revised: x, X ( t + h ) (cid:54) = x }denote the joint event of being in state x (cid:48) (cid:54) =

Original: x at the beginning of the window and in state x at theend, i.e. entering state x between t and t + h .
Revised: x at the beginning of the window and in state x at theend, i.e. entering state x between t and t + h .

Original: Then, we can quantify the ﬂow of probability into xper unit time with φ ( x, t ) ≡ lim h → 0 1 h P (∆ h ( x, t )) .
Revised: Then, we can quantify the ﬂow of probability into xper unit time with φ ( x, t ) ≡ lim h → 0 1 h P (∆ h ( x, t )) .

Original: We call the quantity φ ( x, t ) the input currentof state x at t .
Revised: We call the quantity φ ( x, t ) the input currentof state x at t .

Original: Analogously, we deﬁne the output current by ψ ( x [Equation]
Revised: Analogously, we deﬁne the output current by ψ ( x, t ) ≡ lim h → 0 1 h P ( ∇ h ( x, t )) .

Original: In the following, wemay omit function arguments if they are the same for all functions.
Revised: In the following, we omit function arguments iff they are the same for all functions.

Original: Let themarginal probability of being in state x at t be denoted by p ( x, t )
Revised: Let the marginalprobability of being in state x at t be denoted by p ( x, t )

Original: ≡ P ( X ( t ) =
Revised: ≡ P ( X ( t ) =

Original: x ) .
Revised: x ) .

Original: Then, p isrelated to the input and output currents by
Revised: Then, p is related to theinput and output currents by

Original: [Equation]
Revised: [Equation]

Original: where I ψ ≡ ψ is the identity operator.
Revised: where I ψ ≡ ψ is the identity operator.

Original: This equation is equivalent to the forward equation inan inhomogeneous CTMC and the currents φ ( x, t ) , ψ ( x, t ) can directly be derived from it (seeappendix section A).
Revised: This equation is equivalent to the forward equation inan inhomogeneous CTMC and the currents φ ( x, t ) , ψ ( x, t ) can directly be derived from it (seeappendix section A).

Original: The input and output currents are related by the forward transition operator of the embedded Markov chain φ
Revised: The input and output currents are related by the forward transition operator of the embedded Markov chain φ

Original: = M ψ .
Revised: = M ψ .

Original: The following Markov property makes the currents important w.r.t.. (2).
Revised: The following Markov property makes the currents important w.r.t.. (2).

Original: Let x [ t,t (cid:48) )
Revised: Let x [ t,t (cid:48) )

Original: denote the trajectory of X ( t ) in the interval [ t (cid:48) , t ) .
Revised: denote the trajectory of X ( t ) in the interval [ t (cid:48) , t ) .

Original: Then,lim h → 0 P (cid:0) x [ t,T ) | ∆ h ( x, t ) , x [0 ,t ) (cid:1) =
Revised: Then,lim h → 0 P (cid:0) x [ t,T ) | ∆ h ( x, t ) , x [0 ,t ) (cid:1) =

Original: lim h → 0 P (cid:0) x
Revised: lim h → 0 P (cid:0) x

Original: [ t,T ) | ∆ h ( x, t ) (cid:1) for any x and t .
Revised: [ t,T ) | ∆ h ( x, t ) (cid:1) for any x and t .

Original: Thus, thecurrent φ ( x, t ) speciﬁes the probabilityper unit time of the CTSMC having the Markov property  at tnext to entering state x .
Revised: Thus, thecurrent φ ( x, t ) speciﬁes the probability of the CTSMC having the Markov property per unit time at t a) b) c) d)     5 0 .0 .0 .0 .0 .1 . next to entering state x .

Original: On the other hand ψ ( x, t ) speciﬁes the likelihood of being Markov at t andentering an arbitrary state x (cid:48) (cid:54) = x .
Revised: On the other hand ψ ( x, t ) speciﬁes the likelihood of being Markov at t andentering an arbitrary state x (cid:48) (cid:54) = x .

Original: An illustration of input and output currents is given in Fig.
Revised: An illustration of input and output currents is given in Fig.

Original: In this work, we talk about forward or ﬁltered currents speciﬁcally if we condition on past observations.
Revised: In this work, we talk about forward or ﬁltered currents speciﬁcally if we condition on past observations.

Original: For these, we write φ α ( x, t ) ≡ lim h → 0 1 h P (cid:0) ∆ h ( x, t )
Revised: For these, we write φ α ( x, t ) ≡ lim h → 0 1 h P (cid:0) ∆ h ( x, t )

Original: | y [0 ,t ) (cid:1) and ψ α ( x, t ) ≡lim
Revised: | y [0 ,t ) (cid:1) and ψ α ( x, t ) ≡lim

Original: h → 0 1 h P (cid:0) ∇ h ( x, t ) | y [0 ,t ) (cid:1) .
Revised: h → 0 1 h P (cid:0) ∇ h ( x, t ) | y [0 ,t ) (cid:1) .

Original: We use the notation α ( x, t ) =
Revised: We use the notation α ( x, t ) =

Original: P (cid:0) X ( t ) =
Revised: P (cid:0) X ( t ) =

Original: x | y [0 ,t ) (cid:1) for the a)     5 0 .0 .0 .0 .0 .1 . .b) c) d)
Revised: x | y [0 ,t ) (cid:1) for the forward marginals.

Original: Hidden CTSMC

Original: HSMM approximation forward marginals.

Original: For the forward currents and marginals (3) holds unaltered.
Revised: For the forward currents and marginals (3) holds unaltered.

Original: We complement the forward currents with backward currents φ β ( x, t ) ≡ lim h → 0  P (cid:0) y
Revised: We complement the forward currents with backward currents φ β ( x, t ) ≡ lim h → 0 1 h P (cid:0) y

Original: [ t,T ) | ∆ h ( x, t ) (cid:1) andψ β ( x, t ) ≡ lim h → 0  P (cid:0) y
Revised: [ t,T ) | ∆ h ( x, t ) (cid:1) and ψ β ( x, t ) ≡lim h → 0 1 h P (cid:0) y

Original: [ t,T ) | ∇ h ( x, t ) (cid:1) .
Revised: [ t,T ) | ∇ h ( x, t ) (cid:1) .

Original: We also assume a terminal condition to be ﬁxed at T  .
Revised: We also assume a terminal condition to be ﬁxed at T ∈ R .

Original: Forward and backward currents share important properties.
Revised: Forward and backward currents share important properties.

Original: Let β ( x, t ) =
Revised: Let β ( x, t ) =

Original: P (cid:0) y
Revised: P (cid:0) y

Original: [ t,T ) | X ( t ) =
Revised: [ t,T ) | X ( t ) =

Original: x (cid:1)be a marginal likelihood complementing α ( x, t ) .
Revised: x (cid:1) be amarginal likelihood complementing α ( x, t ) .

Original: Then, we have forα , β and the both currentsd
Revised: Then, we have for the both currents

Original: [Equation]
Revised: [Equation]

Original: while the backward currents are related by ψ β = M † φ β .
Revised: while the backward currents are related by ψ β = M † φ β .

Original: Like (3), a resemblance to classical forwardand backward equations in a CTMC is not coincidental in (4).
Revised: Like (3), a resemblance to classical forwardand backward equations in a CTMC is not coincidental in (4).

Original: We have seen that knowledge ofthe currents φ α , ψ α and ψ β , φ β fully enables calculation of forward and backward terms.
Revised: We have seen that knowledge of thecurrents φ α , ψ α and ψ β , φ β fully enables calculation of forward and backward terms.

Original: In thefollowing, we will outline the necessary equations to determine them.

Original: 2.2 Obtaining the Forward andBackward Currents through Integral Equations
Revised: 2.2 Path Marginalization and Integral Relations

Original: The input and output currents have an interpretation as a marginalization over paths.
Revised: Currents have a path-wise interpretation in the form of a marginalization.

Original: This, andtheir relation to the forward and backward terms (4), enables us to determine how the observationscontribute to α and β in the history dependent CTSMC.
Revised: This, and their relationto the forward and backward terms (4), enables us to determine how the observations contribute toforward and backward terms under history dependence.

Original: In the following, we will use the shorthandx [ t (cid:48) ,t ) =
Revised: In the following, we will use the shorthandx [ t (cid:48) ,t ) =

Original: x to denote the event of the trajectory x
Revised: x to denote the event of the trajectory x

Original: [ t (cid:48) ,t ) being constant in X ( τ ) = x , ∀ τ ∈ [ t (cid:48) , t ) .
Revised: [ t (cid:48) ,t ) being constant in X ( τ ) = x , ∀ τ ∈ [ t (cid:48) , t ) .

Original: We begin by resolving a relationship between p and φ fundamental to CTSMC’s.
Revised: We begin by resolving a relationship between p and φ fundamental to CTSMC’s.

Original: Note, that theprobability per unit time of X ( t ) entering state x at t − τ , τ > 0 , and staying in x at least untilt , can be given by Λ ( τ | x ) φ ( x, t − τ ) =
Revised: Note, that theprobability per unit time of X ( t ) entering state x at t − τ , τ > 0 , and staying in x at least untilt , can be given by Λ ( τ | x ) φ ( x, t − τ ) =

Original: lim h → 0 1 h P (cid:0) x
Revised: lim h → 0 1 h P (cid:0) x

Original: [ t − τ,t ) = x, ∆ h ( x, t − τ ) (cid:1) .
Revised: [ t − τ,t ) = x, ∆ h ( x, t − τ ) (cid:1) .

Original: To obtainp ( x, t ) , we need to marginalize over all trajectories x
Revised: To obtainp ( x, t ) , we need to marginalize over all trajectories x

Original: [ t − τ,t ) having state x at time t .
Revised: [ t − τ,t ) having state x at time t .

Original: Because of the
Revised: Because of the

Original: Markov property at t − τ , we only need to marginalize over all constant trajectories x
Revised: Markov property at t − τ , we only need to marginalize over all constant trajectories x

Original: [0 ,t − τ )
Revised: [0 ,t − τ )

Original: = x .
Revised: = x .

Original: Thus, we are left to integrate over all τ > 0 to obtain p ( x, t ) .
Revised: Thus, we are left to integrate over all τ > 0 to obtain p ( x, t ) .

Original: We stop at τ = t and insert an initialcondition from there.

Original: Doing this, we obtain the fundamental relationship
Revised: Doing this, we obtain the fundamentalrelationship

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where L is a convolutional integral operator deﬁned by L φ ( x, t ) ≡ (cid:82) t 

Revised: where L is a convolutional integral operator deﬁned by L φ ( x, t ) ≡ (cid:82) t

Original: d τ Λ ( t − τ | x ) φ ( x, τ ) .
Revised: d τ Λ ( t − τ | x ) φ ( x, τ ) .

Original: The inhomogeneity g p : X × R ≥ 

Revised: The inhomogeneity g φ : X × R ≥

Original: → R ≥ 0is a known general function and implements the initialcondition.
Revised: → R ≥ 0 implements the initial condition.

Original: Choices for g p will be discussed in section 4.

Original: Instead of asking for x to be kept from t − τat least until t , we may also demand a transition shortly after t to another state to obtain an expression for ψ .
Revised: Instead of asking for x tobe kept from t − τ at least until t , we may also demand a transition shortly after t to another state.

Original: This probability per unit time can then be given by λ ( x, τ ) Λ ( τ | x ) φ ( x, t − τ ) =
Revised: Thisprobability per unit time can then be given by λ ( x, τ ) Λ ( τ | x ) φ ( x, t − τ ) =

Original: f ( τ ) φ ( x, [Equation] − τ ) =
Revised: f ( τ ) φ ( x, t − τ ) =

Original: [Equation] → [Equation] h P (cid:0) ∇ h ( x, t − τ ) , x [ t − τ,t )=
Revised: lim h → 0 1 h P (cid:0) ∇ h ( x, t − τ ) , x [ t − τ,t )

Original:  x, ∆ h ( x, t − τ ) (cid:1) .
Revised: = x, ∆ h ( x, t − τ ) (cid:1) .

Original: As the result of amarginalization over trajectories similar to (5), i.e. integration over τ > 0 , we then obtainψ =
Revised: As the result of a marginalization overtrajectories similar to (5), i.e. integration over τ > 0 , we then obtain [Equation]

Original: F φ + g φ (6) with the integral operator F φ ( x, t ) ≡ (cid:82) t [-
-] d τ f ( t − τ | x ) φ ( x, τ ) .
Revised: [Equation] with the integral operator F φ ( x, t ) ≡ (cid:82) t [-
-]0 d τ f ( t − τ | x ) φ ( x, τ ) .

Original: Again, g φ implements the initialcondition.
Revised: Again, g φ implements the initialcondition.

Original: Note, that we can understand this as an expectation ψ ( x, t ) =
Revised: Note, that we can understand this as an expectation ψ ( x, t ) =

Original: E  [ φ ( x, t − · )] over the waiting time in state x  .
Revised: E F [ φ ( x, t − · )] over thewaiting time measure F ( · | x ) .

Original: Differentiating (5) w.r.t.
Revised: Differentiating (5) w.r.t.

Original: time t and comparing the result with (3) and (6), weget the relation of the inhomogeneities dd t g p + g φ = 0 .
Revised: time t and comparing the result with (3)and (6), we get the relation of the inhomogeneities dd t g p + g φ = 0 .

Original: Remembering that φ = M ψ , we can obtainthe two identities ψ = FM ψ + M  g ψ and φ = MF φ + g φ where g φ = M  g ψ .
Revised: Remembering that φ = M ψ , wecan obtain the two identities ψ = FM ψ + M † g ψ and φ = MF φ + g φ where g φ = M † g ψ .

Original: In the following, we account for observations and then close with the backward currents.
Revised: In the following, we account for observations and then close with the backward currents.

Original: Letυ ( x, t (cid:48) , t ) ≡ P (cid:0) y
Revised: Letυ ( x, t (cid:48) , t ) ≡ P (cid:0) y

Original: [ t (cid:48) ,t ) | x [ t (cid:48) ,t )
Revised: [ t (cid:48) ,t ) | x [ t (cid:48) ,t )

Original: =
Revised: =

Original: x (cid:1) /P
Revised: x (cid:1) /P

Original: (cid:0)
Revised: (cid:0)

Original: y
Revised: y

Original: [ t (cid:48) ,t )
Revised: [ t (cid:48) ,t )

Original: | y [0 ,t (cid:48) )
Revised: | y [0 ,t (cid:48) )

Original: (cid:1) be the normalized joint observationlikelihood of the constant latent trajectory x
Revised: (cid:1) be the normalized joint observationlikelihood of the constant latent trajectory x

Original: [ t (cid:48) ,t )
Revised: [ t (cid:48) ,t )

Original: =
Revised: =

Original: x .
Revised: x .

Original: The normalizer P (cid:0) y
Revised: The normalizer P (cid:0) y

Original: [ t (cid:48) ,t )
Revised: [ t (cid:48) ,t )

Original: | y [0 ,t (cid:48) )
Revised: | y [0 ,t (cid:48) )

Original: (cid:1) is thesame as implicitly used when normalizing in a CTMC ﬁltering update.
Revised: (cid:1)is the same as implicitly used when normalizing in a CTMC ﬁltering update.

Original: The υ ( x, t (cid:48) , t ) is then at -family of piece-wise constant functions with discontinuities in the t (cid:48) -direction
Revised: The υ ( x, t (cid:48) , t )is then a t -family of piece-wise constant functions with discontinuities in the t (cid:48) -direction(or

Original: (or vice-versa).
Revised: vice-versa).

Original: Forconvenience, we deﬁne the modiﬁed integral operator F α φ ( x, t ) ≡  (cid:82) t 0 d τ f υ ( τ, t | x ) φ ( x, τ ) ,where f υ ( τ, t | x ) ≡ υ ( x, τ, t ) f
Revised: For convenience, we deﬁne the modiﬁed integral operator F α φ ( x, t ) ≡ 168 (cid:82) t 0 d τ f υ ( τ, t | x ) φ ( x, τ ) , where f υ ( τ, t | x ) ≡ υ ( x, τ, t ) f

Original: ( t − τ | x ) .
Revised: ( t − τ | x ) .

Original: We also deﬁne L α φ ( x, t ) for Λ υ ( τ, t | x ) ≡υ ( x, τ, t ) Λ ( t − τ | x ) analoguously.
Revised: Deﬁne L α φ ( x, t )for Λ υ ( τ, t | x ) ≡ υ ( x, τ, t ) Λ ( t − τ | x ) analoguously.

Original: Let further υ 0 ( x, t ) ≡ υ ( x, 0 , t ) be a shorthand for υ in theinterval [0 , t ) .
Revised: Let further υ 0 ( x, t ) ≡ υ ( x, 0 , t ) be ashorthand for υ in the interval [0 , t ) .

Original: Then, we can generalize (5) and (6) to obtain
Revised: Then, we can generalize (5) and (6) to obtain

Original: [Equation]
Revised: [Equation]

Original: and 
Revised: and ψ

Original: [Equation]
Revised: [Equation]

Original:  From (7) immediately follow the identities ψ α = F α M ψ α + υ 0 M g ψ and φ α = MF α φ α +υ 0 g φ .
Revised: F [Equation] From (7) immediately follow the identities ψ α = F α M ψ α + υ 0 M g ψ and φ α = MF α φ α +υ 0 g φ .

Original: The backward currents obey relations analogous to (7).
Revised: The backward currents obey relations analogous to (7).

Original: In the following, wewill perform the similar construction.
Revised: In the following, wewill perform the similar construction.

Original: Consider the product f
Revised: Consider the product f

Original: ( τ | x ) ψ β ( x, t + τ ) =
Revised: ( τ | x ) ψ β ( x, t + τ ) =

Original: lim h → 0  P ( y [ t,T ) , ∇ h
Revised: lim h → 0 1 h P ( y [ t,T ) , ∇ h

Original: x, t + τ ) , x [ t,t + τ ) =
Revised: x, t + τ ) , x [ t,t + τ ) =

Original: x | ∆ h ( x, t  )) .
Revised: x | ∆ h ( x, t − τ )) .

Original: It gives the likelihood of keepingstate x in the interval [ t, t + τ ) , then leaving, and along the path encountering any future observations, given an entry in state x at t .
Revised: It gives the probability perunit time of keeping state x in the interval [ t, t + τ ) , then leaving, and along the path encounteringany future observations, all conditioned on entering state x at t .

Original: Thus, φ β ( x, t ) can be obtained again by marginalization overfuture trajectories.
Revised: Thus, φ β ( x, t ) can be obtainedagain by marginalization over future trajectories.

Original: The marginal likelihood β ( x, t ) can also be shown to maintain an analogousrelationship with Λ ( τ | x ) ψ β ( x, t + τ ) .
Revised: The marginal likelihood β ( x, t ) can also beshown to maintain an analogous relationship with Λ ( τ | x ) ψ β ( x, t + τ ) .

Original: This is obtained by a two-side marginalization along stateentry ∆ h ( x, t − τ ) and exit ∇ h ( x, t + τ (cid:48) ) while implying no information on the time of entry t − τ .
Revised: This is obtained by atwo-side marginalization along state entry ∆ h ( x, t − τ ) and exit ∇ h ( x, t + τ (cid:48) ) while implyingno information on the time of entry t − τ .

Original: We refer to appendix section A for the derivation, since it takes more steps.
Revised: We refer to appendix section A for the derivation,since it takes more steps.

Original: In the reverse directionas we did for the forward case (7), we similarly normalize β and the backwards currents ψ β , φ βby P (cid:0) y
Revised: In the reverse direction as we did for the forward case (7), we similarly normalize β and the backwards currents ψ β , φ β by P (cid:0) y

Original: [ t,t (cid:48) ) | y [0 ,t ) (cid:1) .
Revised: [ t,t (cid:48) ) | y [0 ,t ) (cid:1) .

Original: This requires a full forward pass to be performed before the backward calculations.
Revised: This requires a fullforward pass to be performed before the backward calculations.

Original: Deﬁne the integral operators F β ψ ( x, t ) ≡ (cid:82) T t d τ f υ ( t, τ | x ) ψ ( x, τ ) and L β ψ ( x, t )for Λ υ ( t, τ | x ) analogously.
Revised: Deﬁne the integral operators F β ψ ( x, t ) ≡ (cid:82) T t d τ f υ ( t, τ | x ) ψ ( x, τ ) and L β ψ ( x, t ) for Λ υ ( t, τ | x ) analogously.

Original: Let further again υ T ( x, t ) ≡ υ ( x, t, T ) be a shorthand for υ in theinterval [ t, T ) .
Revised: Let further again υ T ( x, t ) ≡ υ ( x, t, T ) be a shorthand for υ in the interval [ t, T ) .

Original: As a conclusion, we obtain the relationsβ =
Revised: As a conclusion, weobtain the relations

Original: L

Original: [Equation]
Revised: [Equation]

Original: ψ [Equation] + υ T γ p and φ β = F β ψ β + υ T γ ψ , (8) where the inhomogeneities γ p and γ ψ implement the terminal condition.
Revised: ) where the inhomogeneities γ p and γ ψ implement the terminal condition.

Original: Note, that we write γ todistinguish from the g for the initial conditions.
Revised: Note, that we write γ todistinguish from the g for the initial conditions.

Original: Again by differentiation and consultation of (4),we can ﬁnd dd t γ p + γ φ = 0 .
Revised: Again by differentiation and consultation of ( ?? ),we can ﬁnd dd t γ p + γ φ = 0 .

Original: Like before, the recurrence relations φ β = M † F β φ β + υ T γ ψ andφ β
Revised: Like before, the recurrence relations φ β = M † F β φ β + υ T γ ψ andφ β

Original: =
Revised: =

Original: F β M † φ β + υ T M † γ φ follow.
Revised: F β M † φ β + υ T M † γ φ follow.

Original: Also, γ ψ = M † γ φ .
Revised: Also, γ ψ = M † γ φ .

Original: Note, that if there are no observations and weonly keep the terminal condition, a particular case of both equations in (8) similar to (5) and (6) isobtained.
Revised: Note, that if there are no observations and weonly keep the terminal condition, a particular case of both equations in (8) similar to (5) and (6) isobtained.

Original:  Integro-Differential Forward and Backward Equations
Revised:  Injecting Observations into Kolmogorov’s Equations

Original: We ﬁrst brieﬂy sketch the derivation of Kolmogorov’s forward equation.
Revised: We ﬁrst brieﬂy sketch the derivation of Kolmogorov’s forward equation.

Original: Our goal is a direct relationbetween dd t p and p .
Revised: Our goal is a direct relationbetween dd t p and p .

Original: We start out with (3) relating dd t p and ψ .
Revised: We start out with (3) relating dd t p and ψ .

Original: We then have (6) expressing ψ in termsof φ and we also have (5) expressing p in terms of φ .
Revised: We then have (6) expressing ψ in termsof φ and we also have (5) expressing p in terms of φ .

Original: Convolutional equations can be convenientlytreated using the Laplace transform.
Revised: Convolutional equations can be convenientlytreated using the Laplace transform.

Original: We use it on (5) to express φ by a convolution of p , then plugthe result into (6) to express ψ in terms of p .
Revised: We use it on (5) to express φ by a convolution of p , then plugthe result into (6) to express ψ in terms of p .

Original: The latter result is then inserted into (3) to express dd t pin terms of p .
Revised: The latter result is then inserted into (3) to express dd t pin terms of p .

Original: The result in the time domain is then
Revised: The result in the time domain is then

Original: [Equation]
Revised: [Equation]

Original: [Equation] τ ) + g ( x, t ) with the integral operator K p ( x, t ) ≡ (cid:82) t [-
-] d τ κ ( x, t − τ ) p ( x, τ ) .
Revised: (9) with the integral operator K p ( x, t ) ≡ (cid:82) t [-
-]0 d τ κ ( x, t − τ ) p ( x, τ ) .

Original: The memory kernel κ is thengiven by L { κ } ( x, s ) ≡ L { f} ( s | x ) (cid:14) L { Λ } ( s | x ) with  the symbol L denoting the Laplacetransform with the Laplace variable s . 
Revised: The memory kernel κ is thengiven by L { κ } ( x, s ) ≡ ˜ f ( s | x ) (cid:14) ˜Λ ( s | x ) with ˜ f and ˜Λ the Laplace transforms of f and Λ withthe Laplace variable s . The symbol L thereby denotes the Laplace transform.

Original: The inhomogeneity g will be discussed at the end of thesection.
Revised: The inhomogeneity gwill be discussed at the end of the section.

Original: In the following, we implement observations and introduce the backward case.
Revised: Before we consider the backward case, we implement observations.

Original: Equation (9) is notsimply understood as apath marginalization  like (5) or (6).
Revised: Equation (9) is not simplyunderstood as a marginalization over paths like (5) or (6).

Original: However,  the path-wise contributionof the observation likelihood carries over from (7) and (8) to equation (9) if the  likelihood functionυ ( x, t (cid:48) , t ) is piecewise constant.The analogue can be shown for the backward case.
Revised: However, we can show that the path-wisecontribution from the observation likelihood carries over from (7) and (8) to equation (9) under thecondition that the observation likelihood function υ ( x, t (cid:48) , t ) is piecewise constant.

Original: The rationale isto subdivide the time-domain into intervals where υ ( x, t (cid:48) , t ) is constant under variation ofeither tin the forward case or t (cid:48) in the backward case.
Revised: The rationale isto subdivide the time-domain into intervals where υ ( x, t (cid:48) , t ) is constant under variation of t .

Original: The two-argument functions ¯ υ ( x, τ ) ≡ υ ( x, τ, t)(forward) and υ ( x, τ ) ≡ υ ( x, t (cid:48) , τ ) (backward) are then  simple stepped functions under applicationof L .
Revised: Thetwo-argument function ¯ υ ( x, τ ) ≡ υ ( x, τ, t ) is then a simple stepped function.

Original: The following derivations are executed in detail in Appendix section A. To obtain the forward equation, we repeat the steps to obtain (9) but instead use our extendedequations (7) for each time interval.
Revised: Under this condition,we repeat the steps to obtain (9) but instead use our extended equations (7) under a time shift. We canthen glue the obtained sequence of equations back together to obtain [Equation]

Revised: with the modiﬁed integral operator K α α ( x, t ) ≡ (cid:82) t 0 d τ κ υ ( τ, t, x ) p ( x, τ ) where κ υ ( x, τ, t ) ≡υ ( x, τ, t ) κ ( x, t − τ ) .

Original: For the backward equation, we do the following steps.
Revised: We conclude with the backward equation. This time we subdivide into timeintervals where υ ( x, t (cid:48) , t ) is constant under variation of t (cid:48) . The two-argument function υ ( x, τ ) ≡υ ( x, t (cid:48) , τ ) is then again a simple stepped function. For each time-interval we do the following steps.

Original: With(4) and the two equations in (8) we have a similar toolset to the forward case.
Revised: With (4) and the two equations in (8) we have a similar toolset to the forward case.

Original: Thus, we apply the Laplace transform to the left equation of (8).
Revised: Thus, we applythe Laplace transform to the left equation of (8).

Original: We then insert ψ β expressed through β into the rightequation of (8) which gets us φ β in terms of β .
Revised: We then insert ψ β expressed through β into the rightequation of (8) which gets us φ β in terms of β .

Original: Inserting this expression into (4) yields the backwardequation valid for the considered time-interval.
Revised: Inserting this expression into (4) yields the backwardequation valid for the considered time-interval .

Original: Joining the obtained sequences of equations backtogether in both cases yields the forward and backward equations [Equation] and
Revised: Again glueing the obtained sequence of equationsback together, we ﬁnally obtaind

Original: [Equation]
Revised: [Equation]

Original: (10) with themodiﬁed integral operators K α α ( x, t ) ≡ (cid:82) t 0 d τ κ υ ( τ, t, x ) p ( x, τ ) and K β β ( x, t ) ≡(cid:82) T t d τ κ υ ( x, t, τ ) β ( x, τ )where κ υ ( x, t (cid:48) , t ) ≡ υ ( x, t (cid:48) , t ) κ ( x, t − t (cid:48) ) .
Revised: (11) with the integral operator K β β ( x, t ) ≡ (cid:82) T t d τ κ υ ( x, t, τ ) β ( x, τ ) .

Original: Both inhomogeneities gand γ can be built from the inhomogeneities g p , g φ and γ p , γ ψ from section 2.2.
Revised: Both inhomogeneities g and γcan be built from the inhomogeneities g p , g φ and γ p , γ ψ from section 2.2.

Original: They satisfy theequations g ≡ ( M − I )
Revised: They satisfy the equationsg ≡ ( M − I )

Original: ( g φ − K α g p ) and γ ≡ (cid:0)
Revised: ( g φ − K α g p ) and γ ≡ (cid:0)

Original: I − M † (cid:1) ( K β γ p − γ ψ ) .
Revised: I − M † (cid:1) ( K β γ p − γ ψ ) .

Original: We will consider these in thenext section.
Revised: We will consider these in the nextsection.

Original: Finally, note that α and β can not simply be multiplied to obtain the full Bayesianposterior w.r.t.
Revised: Finally, note that equations (10) and (11) can not simply be multiplied to obtain the full Bayesian posterior w.r.t.

Original: the observations.
Revised: the observations.

Original:  Latent State Inference
Revised:  Latent State Inference

Original: There are two possibilities to realize the forward and backward algorithms.
Revised: There are two possibilities to realize the forward and backward algorithms.

Original: We exemplify thison the forward algorithm in the following.
Revised: We exemplify this on theforward algorithm in the following.

Original: In popular HSMM literature [22], the discrete-timeequivalent of the products φ α and/or ψ α are fully calculated and α is derived from them using(3).
Revised: In popular HSMM literature [22], the discrete-time equivalent ofthe products φ α and/or ψ α are fully calculated and α is derived from them using (3).

Original: This general procedure poses the computationally most efﬁcient overall solution under twocircumstances.
Revised: This generalprocedure poses the computationally most efﬁcient overall solution under two circumstances.

Original: First, if we also aim to calculate the full Bayesian posterior P (cid:0) X ( t ) = x | y [0 ,T ) (cid:1)for all t we need φ α and/or ψ α anyway.
Revised: First,if we also aim to calculate the full Bayesian posterior P (cid:0) X ( t ) = x | y [0 ,T ) (cid:1) for all t we need φ αand/or ψ α anyway.

Original: Second, if the classical memory kernel K and thus K α is hardto evaluate (e.g. by numerical inverse Laplace transform), an evaluation of F and thus F α might becomputationally preferrable, even if we need to perform an integration afterwards.
Revised: Second, if the classical memory kernel K and thus K α is hard to evaluate (e.g.by numerical inverse Laplace transform), an evaluation of F and thus F α might be computationallypreferrable, even if we need to perform an integration afterwards.

Original: In this work, wechose to evaluate α and β directly from (10)  using a ﬁnite-element approach.
Revised: In this work, we chose to evaluateα and β directly from (10) and (11) using a ﬁnite-element approach.

Original: Thus, we introduce atime-mesh t 0 < t 1 < · · · < t K , where we set each t k exactly at the times of observation.
Revised: Thus, we introduce a time-mesht 0 < t 1 < · · · < t K , where we set each t k exactly at the times of observation.

Original: Initial and Terminal Conditions.
Revised: Boundary Conditions.

Original: Until now, we assumed the boundary conditions g and γ to beknown but didn’t specify them.
Revised: Until now, we assumed the boundary conditions g and γ to be known butdidn’t specify them.

Original: Compared to CTMC’s, the boundary conditions are not single statesbut semi-inﬁnite trajectories.
Revised: Compared to CTMC’s, the boundary conditions are not single states but semiinﬁnite trajectories starting .

Original: In HSMM literature [22], there are two popular boundary conditions.
Revised: In HSMM literature [22], there are two popular boundary conditions.

Original: The ﬁrst assumes, a transition occures both before t 0 and after t K .
Revised: The ﬁrst assumes, a transition occures both before t 0 and after t K .

Original: The second assumes a steadystate before t 0 and an uninformed progression after t K .
Revised: The second assumes a steadystate before t 0 and an uninformed progression after t K .

Original: The latter corresponds to the case that isclassically expressed by a vector of ones in HMM’s when the backward term is initialized.
Revised: The latter corresponds to the case that isclassically expressed by a vector of ones in HMM’s when the backward term is initialized.

Original: More onboundary conditions and their exact expression are found in appendix section A.
Revised: More onboundary conditions and their exact expression are found in appendix section A.

Original: 4.1 Unidirectional Forward and Backward Algorithms
Revised: 4.1 Unidirectional Forward and Backward Algorithms

Original: We formulate the general forward and backward algorithms in appendix section C.
Revised: We formulate the general forward and backward algorithms in appendix section C.

Original: The interpolantsα k and β k valid in the k -th time interval t ∈ [ t k , t k +1 ) obey the following integral equations
Revised: The interpolantsα k and β k valid in the k -th time interval t ∈ [ t k , t k +1 ) obey the following integral equations

Original: [Equation]
Revised: [Equation]

Original: where we introduce the truncated integral operators K [ a,b ) α k ≡ (cid:82) b
Revised: where we introduce the truncated integral operators K [ a,b ) α k ≡ (cid:82) b

Original: a d τ κ ( t − τ, x ) α k ( x, τ ) and
Revised: a d τ κ ( t − τ, x ) α k ( x, τ ) and

Original: K ( a,b ] β k ≡ (cid:82) b a d τ κ ( τ − t, x ) β k ( x, τ ) .
Revised: K ( a,b ] β k ≡ (cid:82) b a d τ κ ( τ − t, x ) β k ( x, τ ) .

Original: The inhomogeneities g k and γ k are built during runtimefrom the overall boundary condition g 0 , the previous interpolants α k (cid:48) , k (cid:48) <
Revised: The inhomogeneities g k and γ k are built during runtime from the overall boundary condition g 0 , the previous interpolants α k (cid:48) , k (cid:48) <

Original: k , and the scalingfunction υ .
Revised: k , and the scalingfunction υ .

Original: The analogue with γ K and β k (cid:48) , k (cid:48) > k holds for the backward case.
Revised: The analogue with γ K and β k (cid:48) , k (cid:48) > k holds for the backward case.

Original: 4.2 Forward-Backward Algorithm - Posterior Marginals
Revised: 4.2 Forward-Backward Algorithm - Posterior Marginals

Original: Obtaining the posterior marginals is possible on the level of currents.
Revised: Obtaining the posterior marginals is possible on the level of currents.

Original: Since they mark times when the
Revised: Since they mark times when the

Original: CTSMC has the Markov property, we can build products of the two normalized currents ψ α ( x, t ) =
Revised: CTSMC has the Markov property, we can build products of the two normalized currents ψ α ( x, t ) =

Original: lim h → 0 1 h P ( ∇ h ( x, t ) | y [0 ,t ) ) and ψ β ( x, t ) =
Revised: lim h → 0 1 h P ( ∇ h ( x, t ) | y [0 ,t ) ) and ψ β ( x, t ) =

Original: lim h → 0 1 h P ( y [ t,T ) | ∇ h ( x, t ))
Revised: lim h → 0 1 h P ( y [ t,T ) | ∇ h ( x, t ))

Original: / P ( y [ t,T ) |y [0 ,t ) ) to obtain the posterior current ˆ ψ ( x, t ) =
Revised: / P ( y [ t,T ) |y [0 ,t ) ) to obtain the posterior current ˆ ψ ( x, t ) =

Original: lim h → 0 1 h P ( ∇ h ( x, t ) |
Revised: lim h → 0 1 h P ( ∇ h ( x, t ) |

Original: y
Revised: y

Original: [0 ,T ) ) .
Revised: [0 ,T ) ) .

Original: which we canuse in (3) to obtain the derivative dd t ˆ p .
Revised: which we can usein (3) to obtain the derivative dd t ˆ p .

Original: However, we discuss this approach in appendix section A. Although it being preferable from a runtime perspective, an more classical approachpopular indiscrete-time HSMM literature [22] uses a two-sided path marginalization which allows usage ofinterpolation-based numerical methods.
Revised: However, we discuss this approach in appendix section C. Althoughit being preferable from a runtime perspective, the approach is more susceptible to numerical errors. A numerically preferable and more classical approach uses a two-sided path marginalization and ispopular in discrete-time HSMM literature [22].

Original: In this work, we calculate the posterior marginals usingˆ p ( x,
Revised: Thus, we suggest calculating the posterior marginalsusing [Equation]

Original: [Equation]
Revised: [Equation]

Original: We need to add that both currents φ α and ψ β are obtained as by-products without additional computation from the unidirectional forward and backward algorithms introduced in section 4.1.
Revised: Note that both currents φ α and ψ β are obtained as by-products without additional computation fromthe unidirectional forward and backward algorithms introduced in section 4.1.

Original: Although(12) is a two-dimensional integral and therefor scales quadratically with T , it is linear in |X| sinceonly components of one state occur.
Revised: Although (13) isa two-dimensional integral and therefor scales quadratically with T , it is linear in |X| since onlycomponents of one state occur.

Original: An algorithmic descriptionand an explanation how φ α and ψ βare obtained as by-products can be found in appendix section C.
Revised: An algorithmic description can be found in appendix section D.

Original: 4.3 Viterbi-type Algorithm - Latent Trajectory Point Estimation
Revised: 4.3 Viterbi-type Algorithm - Latent Trajectory Point Estimation

Original: Besides latent state marginals, point estimates of whole posterior paths are as well important in furtherinference tasks.
Revised: Besides latent state marginals, point estimates of whole posterior paths are as well important in furtherinference tasks.

Original: Originally stemming from HMM’s, the Viterbi algorithm which calculates MAP pathestimates has been successfully adopted to HSMM’s
Revised: Originally stemming from HMM’s, the Viterbi algorithm which calculates MAP pathestimates has been successfully adopted to HSMM’s

Original: However, like in hidden CTMC’s, theunknown chain length, i.e. the number of state changes until the terminal state is reached, makes thealgorithm hard to translate to continuous time.
Revised: However, like in hidden CTMC’s, theunknown chain length, i.e. the number of state changes until the terminal state is reached, makes thealgorithm hard to translate to continuous time.

Original: One is therefor often content with knowing the MAPstate only at observation times [14].
Revised: One is therefor often content with knowing the MAPstate only at observation times [14].

Original: The latter can in principle be calculated for hidden CTSMC’susing our forward equation (9).
Revised: The latter can in principle be calculated for hidden CTSMC’susing our forward equation (9).

Original: Nonetheless, for a ﬁnite number of jumps, the framework of currentsallows the formulation of a scalable full-path algorithm.
Revised: Nonetheless, for a ﬁnite number of jumps, the framework of currentsallows the formulation of a scalable full-path algorithm.

Original: A max-product-type reformulation of (7)allows an introduction of a max-input current
Revised: A max-product-type reformulation of (x)allows an introduction of a max-input current

Original: [Equation]
Revised: [Equation]

Original: which - bluntly speaking - substitutes the marginalization in φ α with a supremum.
Revised: which - bluntly speaking - substitutes the marginalization in φ α with a supremum.

Original: To keeptrack of the jump count, we equip this current with an additional parameter n .
Revised: To keeptrack of the jump count, we equip this current with an additional parameter n .

Original: We also introduce a maximizing transition operator M max for the embedded Markov chain M max g ( x, t ) ≡max x 
Revised: We also introduce a maximizing transition operator M max for the embedded Markov chain M max g ( x, t ) ≡max x [Equation] = [Equation]

Revised: m

Original: [Equation]
Revised: [Equation]

Original: xm ( x | x (cid:48) ) g ( x (cid:48) , t ) .
Revised: x | x [Equation] ) g ( x (cid:48) , t ) .

Original: The transition operator M max resembles M where the sum over theset X \ { x } is exchanged by a maximization.
Revised: The transition operator M max resembles M where the sum over theset X \ { x } is exchanged by a maximization.

Original: The constituting recursion for φ max ( x, t ; n ) is thengiven by
Revised: The constituting recursion for φ max ( x, t ; n ) is thengiven by

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: which calculates the max-currents φ max ( x, t ; n + 1) associated with the incremented chain lengthn + 1 .
Revised: which calculates the max-currents φ max ( x, t ; n + 1) associated with the incremented chain lengthn + 1 .

Original: A detailed description  is found in appendix section A. The MAP terminal state x is related tothe currents by
Revised: A detailled description and a comparison to the original Viterbi algorithm is found in appendixsection A. The MAP terminal state x is related to the currents by

Original: [Equation]
Revised: [Equation]

Original: Let us ignore the chain length marginal P ( n ) for now and assume we found a maximizing tuple( x ∗ , τ ∗ , n ∗ ) .
Revised: Let us ignore the chain length marginal P ( n ) for now and assume we found a maximizing tuple( x ∗ , τ ∗ , n ∗ ) .

Original: Then, if n ∗ > 0 , we can insert φ max ( x ∗ , τ ∗ ; n ∗ ) in (13) and proceed to trace backstates and transition instances until the jump counter reaches zero.
Revised: Then, if n ∗ > 0 , we can insert φ max ( x ∗ , τ ∗ ; n ∗ ) in (4.3) and proceed to trace backstates and transition instances until the jump counter reaches zero.

Original: However, to ﬁnd the tuple( x ∗ , τ ∗ , n ∗ ) , we need knowledge of at least some scaling factor L ( n ) ∝ P ( n ) .
Revised: However, to ﬁnd the tuple( x ∗ , τ ∗ , n ∗ ) , we need knowledge of at least some scaling factor L ( n ) ∝ P ( n ) .

Original: Proper normalization to get P ( n ) requires knowledge of n up to inﬁnity, which is intractable.
Revised: Proper normalization to get P ( n ) requires knowledge of n up to inﬁnity, which is intractable.

Original: However, a scalingfactor L ( n ) can be found recursively.
Revised: However, a scalingfactor L ( n ) can be found recursively.

Original: To do this, two more sets of input currents φ n ( x, t ) ≡lim h → 0 1 h P (cid:0) ∆ h ( x, t ) , y [0 ,t ) , n (cid:1) and marginals p n ( x, T ) ≡ P (cid:0) X ( T )=
Revised: To do this, two more sets of input currents φ n ( x, t ) ≡ lim h → 0 1 h P (cid:0) ∆ h ( x, t ) , y [0 ,t ) , n (cid:1) and marginals p n ( x, T ) ≡ P (cid:0) X ( T )

Original:  x, y [0 ,t ) , n (cid:1) are introduced.
Revised: = x, y [0 ,t ) , n (cid:1) are introduced.

Original: The φ n ( x, t ) and p n ( x, T ) are then updated by
Revised: The φ n ( x, t ) and p n ( x, T ) are then updated by

Original: [Equation]
Revised: [Equation]

Original: n and

Original: [Equation]
Revised: [Equation]

Original: The currents are initialized by φ 0 ( x, t ) =
Revised: The currents are initialized by φ 0 ( x, t ) =

Original: υ ( x, 0 , t ) g φ ( x, t ) .
Revised: υ (0 , t ) g φ ( x, t ) .

Original: Note that the left equation in (14)does not converge to the ﬁxed-point φ α
Revised: Note that (14) does not converge tothe ﬁxed-point φ α

Original: but instead will vanish for n → ∞ .
Revised: but instead will vanish for n → ∞ .

Original: The factors L ( n ) ≡ P (cid:0) y [0 ,t ) , n (cid:1) =
Revised: The factors L ( n ) ≡ P (cid:0) y [0 ,t ) , n (cid:1) =

Original:  (cid:80) x p n ( x, T ) can then be obtained by marginalization.
Revised: 294 (cid:80) x p n ( x, T ) can then be obtained by marginalization.

Original: Again, we refer to appendixsection A. Unfortunately, to obtain an exact result, we still need to evaluate all equations for arbitrarilylarge n .
Revised: Again, we refer to appendix section A. Unfortunately, to obtain an exact result, we still need to evaluate all equations for arbitrarily large n .

Original: For most practical scenarios, however, the most likely number of transitions n ∗ might bemanageable.
Revised: For most practical scenarios, however, the most likely number of transitions n ∗ might be manageable.

Original: We stress that the bottleneck of our algorithm is the number of transitions, not theirtemporal closeness.
Revised: We stress that the bottleneck of our algorithm is the number of transitions, not their temporal closeness.

Original:  Adaptive Step-Size HSMM
Revised:  Adaptive Step-Size HSMM

Original: One of the major advantages of continuous-time formulations is the availability of adaptive discretization [8, 4].
Revised: One of the major advantages of continuous-time formulations is the availability of adaptive discretization [8, 4].

Original: Although many continuous problems eventually need to be solved numerically on adiscrete grid, adaptive step size methods can bound the discretization error and allow a reduction ofthe amount of function evaluations.
Revised: Although many continuous problems eventually need to be solved numerically on adiscrete grid, adaptive step size methods can bound the discretization error and allow a reduction ofthe amount of function evaluations.

Original: While much more elaborate techniques with tight error boundsare available for integral equations [10], we brieﬂy sketch a simple adaptive discretization strategy toclose with a practical perspective on the contributions in this work.
Revised: While much more elaborate techniques with tight error boundsare available for integral equations [10], we brieﬂy sketch a simple adaptive discretization strategy toclose with a practical perspective on the contributions in this work.

Original: This strategy consists of maintaining a time-dependent set of samples of approximate currents andtheir timestamps Φ α ( t ) ≡ { ( φ α,k , t k ) | t K − t 0 < ∆ max , t 0 < · · · < t K , k = 0 , . .
Revised: This strategy consists of maintaining a time-dependent set of samples of approximate currents andtheir timestamps Φ α ( t ) ≡ { ( φ α,k , t k ) | t K − t 0 < ∆ max , t 0 < · · · < t K , k = 0 , . .

Original: . , K } .
Revised: . , K } .

Original: Ananalogue set is built for ψ β .
Revised: Ananalogue set is built for ψ β .

Original: Then, by inserting polynomial interpolants (e.g. afﬁne combinationsof adjacent ( φ α,k , t k ) and ( φ α,k +1 , t k +1 ) ), we can reformulate (7) so that it only contains the φ α,k , 307 t k and deﬁnite integrals of the f ( · | x ) .
Revised: Then, by inserting polynomial interpolants (e.g. afﬁne combinationsof adjacent ( φ α,k , t k ) and ( φ α,k +1 , t k +1 ) ), we can reformulate (7) so that it only contains the φ α,k , 309 t k and deﬁnite integrals of the f ( · | x ) .

Original: For these integrals, look-up tables are easily generated forspeedup.
Revised: For these integrals, look-up tables are easily generated forspeedup.

Original: The resulting linear system can then be solved to calculate the next step φ (1)
Revised: The resulting linear system can then be solved to calculate the next step φ (1)

Original: α ( x, t K + h ) .
Revised: α ( x, t K + h ) .

Original: As a commonly adapted update rule for step size h , we use half step checks φ (0 . 5)
Revised: As a commonly adapted update rule for step size h , we use half step checks φ (0 . 5)

Original: α ( x, t K + h ) , wherewe do two steps using h 2 as step size and then calculate E ≡ | φ (1)
Revised: α ( x, t K + h ) , wherewe do two steps using h 2 as step size and then calculate E ≡ | φ (1)

Original: α ( x, t K + h ) − φ (0 . 5)
Revised: α ( x, t K + h ) − φ (0 . 5)

Original: α ( x, t K + h ) | .
Revised: α ( x, t K + h ) | .

Original: The integration error E is then used in the update h → max { min { hs, h max , t (cid:48) − t K − h } , h min }with the scaling s = max { min { tol E , s max } , s min } for some pre-speciﬁed h min , h max , s min , s maxand the next observation time t (cid:48) .
Revised: The integration error E is then used in the update h → max { min { hs, h max , t (cid:48) − t K − h } , h min }with the scaling s = max { min { tol E , s max } , s min } for some pre-speciﬁed h min , h max , s min , s maxand the next observation time t (cid:48) .

Original: The resulting HSMM is then only dependent on time-differences, i.e. still homogeneous, but adaptsits step-size.
Revised: The resulting HSMM is then only dependent on time-differemces, i.e. still homogeneous, but adaptsits step-size.

Original: In appendix section C, we provide the resulting update equation if afﬁne combinationsare used as interpolants between the points ( φ α,k , t k ) and an implicit Euler step rule is chosen.
Revised: In appendix section C, we provide the resulting update equation if afﬁne combinationsare used as interpolants between the points ( φ α,k , t k ) and an implicit Euler step rule is chosen.

Original:  Evaluation
Revised:  Evaluation

Original: Calculation of Integral Equations. Our algorithms demand the solution of integral equations of thesecond kind.
Revised: Our algorithms demand the solution of integral equations of the second kind with discontinuities.

Original: For each time interval [ t, t (cid:48) ) between observations, we used a  backward Euler methodto solve the possibly stiff equationsin (7),  (8) or (10) with a ﬁxed number of steps calculated by
Revised: For each time interval [ t, t (cid:48) ) between observations, we used a basic backward Euler method tosolve the possibly stiff equations (7), (10), (8) or (11) with a ﬁxed number of steps calculated by

Original: N ≡ (cid:98) t (cid:48)
Revised: N ≡ (cid:98) t (cid:48)

Original: − t h (cid:99) + 1 .
Revised: − t h (cid:99) + 1 .

Original: Then, all steps are of equal length h except the one at the end of the interval whichscales dependent on t (cid:48) .
Revised: Thus, all steps are of equal length with exception of those at the end of an intervalwhich scales dependent on t (cid:48) .

Original: The integration involved in each step is performed by interpolation betweenthe steps, where the trapezoidal rule has been used.

Original: This method is outlined in detail in Appendixsection B.

Original: An exception is the evaluation of the adaptive step-size HSMM.
Revised: An exception is the evaluation of the adaptive step-size HSMM.

Original: As described in section 5,there the step size is initialized by a smallest value 10 − 4 and then the adaptive procedure generatesany further grid positions.
Revised: There,as described in section 5, the step size is initialized by the smallest value 10 − 4 and then the adaptiveprocedure generates any further grid instances.

Original: The integration at each step is also more precise. The details are outlinedin
Revised: The integration itself is performed by interpolationbetween the steps, where the trapezoidal rule has been used.

Original: Appendix B. For forward and backward passes, each interval is alculated consecutively while theinhomogeneities g and γ are updated upon entering each new interval as outline in the algorithmsection in Appendix C.
Revised: For forward an backward passes, eachinterval is then calculated consecutively while the inhomogeneities g and γ are updated upon enteringeach new interval.

Original: The resulting continuous functions α , β , ˆ p and the currents are then generatedfor each interval using cubic spline interpolation on the calculated steps.
Revised: The calculated functions α , β , ˆ p and the currents were generated using cubicspline interpolation. The exact procedure is detailled in appendix section D.

Original: Experiments. We calculated several randomly generated hidden CTSMC’s.
Revised: We calculated several randomly generated hidden CTSMC’s specifying the set of states.

Original: The number of stateshas been given, but waiting time distributions and transition probabilities were drawnrandomly. Foreach state, we drew a randomlyparametrized waiting time distribution from a speciﬁed family ofdistributions (we allowed Gamma and Weibull).
Revised: Waitingtimes where drawn randomly from a given family of distributions and transition operators of theembedded Markov chains were sampled as sets of random categorical distributions on the state space.      30 0 .0 .a) b)     5    5 c) d)     10         8 10 12 14 0.

Original: The random parameters, were drawn from Gammadistributions with per-parameter calibrated hyperparameters.
Revised: 1.2 1e For each waiting time family, we had a Gamma prior over each parameter with per-family calibratedhyperparameters.

Original: The transition probabilities of theembedded Markov chains were sampled as sets of random categorical distributions while prohibitingself-transitions.
Revised: The categorical distributions were sampled drawing a sequence of iid multinomialoutcomes of which ensemble frequencies were calculated to generate a categorical mass function.

Original: The categorical distributions were sampled by ﬁrst drawing a vector uniformly fromthe |X| -hypercube and then projecting it to the ( |X| − 1) -simplex via normalization. The observationtimes were drawn from apoint process with Gamma inter-arrival times with ﬁxed shape and rate
Revised: The observation times were drawn from a Gamma process with hyperparameters shape and rate,while the observation values were noisy state observations Y ( t ) N ( X ( t ) , σ ) with the std σ a givenhyperparameter.

Original: [-1.2 d) c) b) a) 10    10            0 2 4 6 8 10 
-]
Revised: [-1.2 d) c) b) a) 10    10            0 2 4 6 8 10 
-]Details are found in appendix section E.

Original: The calculated α , β , ˆ p were compared to those calculated by a HSMM.
Revised: The calculated α , β , ˆ p were compared to those calculated by a HSMM.

Original: With a step-size of h HSMM =10 − 4 and h CTSMC = 10 − 3 (backward Euler step-size), no difference was visible anymore on anycalculated trajectory.
Revised: With a step-size of h HSMM =10 − 4 and h CTSMC = 10 − 3 , no difference was visible anymore on any calculated trajectory.

Original: InFig.
Revised: In

Original: 2a, we show an exemplary progression of the forward input and backwardoutput currents.

Original: In Fig. 
Revised: Fig. 2.b

Original: 2b the smoothed marginals ˆ p of a four-state CTSMC are compared to thoseobtained from its steady-state CTMC approximation [13].
Revised: the smoothed marginals ˆ p of a four-state CTSMC are compared to those obtained from itssteady-state CTMC approximation [13].

Original: For this, smoothed mean and varianceof b ( X ( t )) are shown.

Original: The full trajectory of b ( X ( t )) by the original CTSMC which generated theobservations is shown as a ground truth.

Original: We built the adaptive step-size HSMM using our proposedmethod and ran it over the sampled trajectories as well.
Revised: We built the adative step-size HSMM using our proposedmethod and ran it over the sampled trajectories as well.

Original: While quickly enlarging its step-size, itremained within the margins of a total error of 10 − 3 compared to a ﬁne-grained HSMM, althoughthis is not guaranteed by the method.
Revised: While quickly enlarging its step-size, itremained within the margins of a total error of 10 − 3 compared to a ﬁne-grained HSMM althoughnot guaranteed by the method.

Original: An example run is shown in Fig. 2c.
Revised: An example run is shown in Fig. 2.c.

Original: Finally, we tested our 
Revised: Finally, we testet our Viterbi

Original: Viterbi MAP estimator on the trajectories.
Revised: MAP estimator on the trajectories.

Original: An example run is shown in 2d.
Revised: An example run is shown in 2.d.

Original: The chain length scalingfactor L ( n ) ∝ P ( n ) had a comparably sharp peak in all trajectories.
Revised: The chain length scaling factor L ( n ) ∝ P ( n ) had a comparably sharp peak in all trajectories (cf.

Revised: appendix section D).

Original: We assume, this came from theobservations greatly narrowing down the amount of candidate transition counts.
Revised: We assume,this came from the observations greatly narrowing down the amount of candidate transition counts.

Original:  Conclusion
Revised:  Conclusion

Original: We have presented a forward-backward framework applicable to hidden CTSMC’s.
Revised: We have presented a forward-backward framework applicable to hidden CTSMC’s.

Original: Avoidîng statespace augmentation to enforce the Markov property, it gives rise to integral and integro-differentialforward and backward equations and a scalable Viterbi algorithm.
Revised: Avoidîng statespace augmentation to enforce the Markov property, it gives rise to integral and integro-differentialforward and backward equations and a scalable Viterbi algorithm.

Original: Adaptive step-size HSMM’scan be derived, overcoming discrete-time core problems: out-of-grid observations and steady-stateover-computation.
Revised: Adaptive step-size HSMM’scan be derived, overcoming discrete-time core problems: out-of-grid observations and steady-stateover-computation.

Original: We didn’t consider CTSMC’s not satisfying direction-time independence whichare especially interesting for coarse-graining of large Markovian systems.
Revised: We didn’t consider CTSMC’s not satisfying direction-time independence whichare especially interesting for coarse-graining of large Markovian systems.

Original: Also, speciﬁc classes ofwaiting times could allow efﬁcient computation of currents, e.g. power-law waiting times related tofractional calculus being of recent interest in physics.
Revised: Also, speciﬁc classes ofwaiting times could allow efﬁcient computation of currents, e.g. power-law waiting times related tofractional calculus being of recent interest in physics.

Original: Possible solutions would then ﬁt naturally in
Revised: Possible solutions would then ﬁt naturally inour framework which handles observation likelihood of past events.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .
Revised: [Yes] See Section ?? .

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.•
Revised: [No] The code and the data areproprietary.•

Original: Did you include the license to the code and datasets?
Revised: Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A](d)

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]
Revised: [Yes]2.

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes](b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] In Appendix A3.
Revised: [Yes] In Appendix A3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: In theSupplement
Revised: In theSupplement

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experi- ments multiple times)?
Revised: g., with respect to the random seed after running experiments multiple times)?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you include the total amount of compute and the type of resources used (e
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original: g., type of GPUs, internal cluster, or cloud provider)?
Revised: g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] In Appendix E
Revised: [Yes] In Appendix E4.

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c ) Did you include any new assets either in the supplemental material or as a URL?
Revised: c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d ) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e ) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]