Original: Tractable Latent State Inference for Hidden Continuous-Time semi-Markov Chains
Revised: Tractable Latent State Inference for Hidden Continuous-Time semi-Markov Chains

Original: Abstract
Revised: Abstract

Original: Hidden semi-Markov Models (HSMM’s) - while broadly in use - are restricted toa discrete and uniform time grid.
Revised: Hidden semi-Markov Models (HSMM’s) - while broadly in use - are restricted toa discrete and uniform time grid.

Original: They are thus not well suited to explain oftenirregularly spaced discrete event data from continuous-time phenomena.
Revised: They are thus not well suited to explain oftenirregularly spaced discrete event data from continuous-time phenomena.

Original: We showthat non-sampling-based latent state inference used in HSMM’s can be generalizedto latent Continuous-Time semi-Markov Chains (CTSMC’s).
Revised: We showthat non-sampling-based latent state inference used in HSMM’s can be generalizedto latent Continuous-Time semi-Markov Chains (CTSMC’s).

Original: We formulate integrodifferential forward and backward equations adjusted to the observation likelihoodand introduce an exact integral equation for the Bayesian posterior marginals and ascalable Viterbi-type algorithm for posterior path estimates.
Revised: We formulate integrodifferential forward and backward equations adjusted to the observation likelihoodand introduce an exact integral equation for the Bayesian posterior marginals and ascalable Viterbi-type algorithm for posterior path estimates.

Original: All presented equationscan be efﬁciently solved using existing numerical methods.
Revised: All presented equationscan be efﬁciently solved using existing numerical methods.

Original: We evaluate ourapproaches in latent state inference scenarios in comparison to classical HSMM’s.
Revised: We evaluate ourapproaches in latent state inference scenarios in comparison to classical HSMM’s.

Original:  Introduction
Revised:  Introduction

Original: Continuous-Time semi-Markov Chains (CTSMC’s) comprise the continuous time analogue of(discrete-time) semi-Markov chains.
Revised: Continuous-Time semi-Markov Chains (CTSMC’s) comprise the continuous time analogue of(discrete-time) semi-Markov chains.

Original: In comparison to Markov chains, semi-Markov chains allow to model non-exponential, arbitrary waiting times between state changes.
Revised: In comparison to Markov chains, semi-Markov chains allow to model non-exponential, arbitrary waiting times between state changes.

Original: This more accuratelymatches such times occurring in many scientiﬁc ﬁelds [11].
Revised: This more accuratelymatches such times occurring in many scientiﬁc ﬁelds [11].

Original: Hidden semi-Markov Models (HSMM’s)on ﬁnite state spaces have seen wide application and thorough research [22].
Revised: Hidden semi-Markov Models (HSMM’s)on ﬁnite state spaces have seen wide application and thorough research [22].

Original: However, confrontedwith measurements from continuous-time systems, their adaptivity is limited and computation mightbecome unnecessarily expensive.
Revised: However, confrontedwith measurements from continuous-time systems, their adaptivity is limited and computation mightbecome unnecessarily expensive.

Original: Because of that, continuous time analogues of HSMM’s have seenincreased research interest lately [17, 1, 6, 7].
Revised: Because of that, continuous-time analogues of HSMM’s have seenincreased research interest lately [17, 1, 6, 7].

Original: However, latent state inference is hard in continuoustime, since the usual methods demand a reformulation of the process to obtain the Markov property.
Revised: However, latent state inference is hard in continuoustime, since the usual methods demand a reformulation of the process to obtain the Markov property.

Original: Markovianization by state space expansion as used for HSMM’s demands propagation of asemi-inﬁnite uncountable state for latent CTSMC’s
Revised: Markovianization by state space expansion as used for HSMM’s demands propagation of asemi-inﬁnite uncountable state for latent CTSMC’s

Original: even if the original state space is ﬁnite.
Revised: even if the original state space is ﬁnite.

Original: Thus,besides ad-hoc discretization, two classes of solution methods have emerged.
Revised: Thus,besides ad-hoc discretization, two classes of solution methods have emerged.

Original: First, a variety ofsampling-based solutions has been published, often targeted at even generalized problems [7, 2, 1, 9].
Revised: First, a variety ofsampling-based solutions has been published, often targeted at even generalized problems [7, 2, 1, 9].

Original: Second, if non-sampling based solutions are sought, state-space expansions based on approximationsof waiting time measures are available [16].
Revised: Second, if non-sampling based solutions are sought, state-space expansions based on approximationsof waiting time measures are available [16].

Original: This second class, by detour, essentially tries to approximate Kolmogorov’s (forward/backward) equations.
Revised: This second class, by detour, essentially tries to approximate Kolmogorov’s (forward/backward) equations.

Original: However, a general framework - not tailoredto speciﬁc families of waiting time measures - is still lacking.
Revised: However, a general framework - not tailoredto speciﬁc families of waiting time measures - is still lacking.

Original: Kolmogorov’s equations in integrodifferential form have been studied recently [20].
Revised: Kolmogorov’s equations in integrodifferential form have been studied recently [20].

Original: Although initial or terminal states are given, theequations rely on speciﬁc boundary conditions to restrict the non-Markovian process to the consideredtime-window.
Revised: Although initial or terminal states are given, theequations rely on speciﬁc boundary conditions to restrict the non-Markovian process to the consideredtime-window.

Original: Fixed conditions hinder the equations’ usage in latent state inference which typicallyspans many such time-windows.
Revised: Fixed conditions hinder the equations’ usage in latent state inference which typicallyspans many such time-windows.

Original: To alleviate this restriction, we show how to adjust the memoryterms in Kolmogorov’s equations to include observations.
Revised: To alleviate this restriction, we show how to adjust the memoryterms in Kolmogorov’s equations to include observations.

Original: As a consequence, we provide generalforward and a backward algorithms, each encoded in a single inhomogeneous integro-differentialequation.
Revised: As a consequence, we provide generalforward and a backward algorithms, each encoded in a single inhomogeneous integro-differentialequation.

Original: We also provide a method for exact Bayesian smoothing which uses the quantities obtainedfrom the forward and backward algorithms.
Revised: We also provide a method for exact Bayesian smoothing which uses the quantities obtainedfrom the forward and backward algorithms.

Original: As a corollary, we propose a Viterbi-type approach formaximum-a-posteriori path estimation and sketch a method to implement adaptive step-size HSMM’s that keep their homogenous properties.
Revised: As a corollary, we propose a Viterbi-type approach formaximum-a-posteriori path estimation and sketch a method to implement adaptive step-size HSMM’s that keep their homogenous properties.

Original: The resulting general framework may incorporate arbitrarynumerical solversavailable or make use of approximations by the mentioned second class of solutionmethods.
Revised: The resulting general framework may incorporate arbitrarynumerical solvers or make use of approximations by the mentioned second class of solution methods.

Original: Related Work.
Revised: Related Work.

Original: Continuous-time HMM’s and their inference has recently been compiled by [14]).
Revised: Continuous-time HMM’s and their inference has recently been compiled by [14]).

Original: Although tailored to a speciﬁc problem, an interesting MAP path estimation method for CT-HSMM’sbased on stochastic state classes has been published in [5].
Revised: Although tailored to a speciﬁc problem, an interesting MAP path estimation method for CT-HSMM’sbased on stochastic state classes has been published in [5].

Original: Neural ODE’s for the progression ofmemory variables have been investigated in [7]).
Revised: Neural ODE’s for the progression ofmemory variables have been investigated in [7]).

Original: An application of CT-HSMM’s including parameterinference using a discretized waiting time variable [6, 17].
Revised: An application of CT-HSMM’s including parameterinference using a discretized waiting time variable [6, 17].

Original: Generalized HSMM’s that are losslesscontinuous discretizations but the transitions are observable has been proposed in [21].
Revised: Generalized HSMM’s that are losslesscontinuous discretizations but the transitions are observable has been proposed in [21].

Original: An elaborateforward-ﬁltering-backward-sampling for a HASMM [1].
Revised: An elaborateforward-ﬁltering-backward-sampling for a HASMM [1].

Original: To our knowledge, we are the ﬁrst tointroduce a "traditional" form of a forward-backward algorithm for a latent CTSMC.
Revised: To our knowledge, we are the ﬁrst tointroduce a "traditional" form of a forward-backward algorithm for a latent CTSMC.

Original: Also, while [ 5 ]is able to do MAP estimates of trajectories as well, their method is very different to our approach.
Revised: Also, while [ 5 ]is able to do MAP estimates of trajectories as well, their method is very different to our approach.

Original: Notation.
Revised: Notation.

Original: To maintain readability, wewill use the symbol P to denote density and mass functionsalike.
Revised: To maintain readability, we use the symbol P to denote density and mass functions alike.

Original: Which kind is needed should be clear from the context.
Revised: Which kind is needed should be clear from the context.

Original: Nonetheless, we refer to appendixsection A for derivations and proofs using a stricter notation style.
Revised: Nonetheless, we refer to appendix section Afor derivations and proofs using a stricter notation style.

Original: 1.1 Continuous-Time semi-Markov Chains
Revised: 1.1 Continuous-Time semi-Markov Chains

Original: A continuous-time semi-Markov chain (CTSMC) { X ( t ) } t ≥ 0 on a countable state space X is astochastic jump process.
Revised: A continuous-time semi-Markov chain (CTSMC) { X ( t ) } t ≥ 0 on a countable state space X is astochastic jump process.

Original: The general process is described by transition rates λ : X × R ≥ 

Revised: The general process is described by transition rates λ : X × R ≥

Original: × R ≥ 0 ×
Revised: × R ≥ 0 ×

Original: X → R ≥ 0 .
Revised: X → R ≥ 0 .

Original: Conditioned on the current state x ∈ X at time t ∈ R ≥ 0 and the time τ since the lasttransition at t − τ < t , the rates λ ( x, τ, t ; x (cid:48) ) describe the probability per unit time of transitioningshortly after time t to state x (cid:48) .
Revised: Conditioned on the current state x ∈ X at time t ∈ R ≥ 0 and the time τ since the lasttransition at t − τ < t , the rates λ ( x, τ, t ; x (cid:48) ) describe the probability per unit time of transitioningshortly after time t to state x (cid:48) .

Original: These transition rates sufﬁce to resolve the waiting times and jumpdirections for each tuple ( x, τ ) at all times t .
Revised: These transition rates sufﬁce to resolve the waiting times and jumpdirections for each tuple ( x, τ ) at all times t .

Original: In this work, we restrict ourselves to CTSMC’s which arehomogeneous and direction-time independent.
Revised: In this work, we restrict ourselves to CTSMC’s which arehomogeneous and direction-time independent.

Original: The former means that λ ( x, τ, t ; x (cid:48) )
Revised: The former means that λ ( x, τ, t ; x (cid:48) )

Original: ≡ λ ( x, τ ; x (cid:48) )
Revised: ≡ λ ( x, τ ; x (cid:48) )

Original: at all times t .
Revised: at all times t .

Original: The latter additionally implies the factorization λ ( x, τ ; x (cid:48) )
Revised: The latter additionally implies the factorization λ ( x, τ ; x (cid:48) )

Original: ≡ m ( x (cid:48) | x ) λ ( x, τ )with the exit rate λ ( x, τ ) ≡ (cid:80) x (cid:48) (cid:54) =
Revised: ≡ m ( x (cid:48) | x ) λ ( x, τ )with the exit rate λ ( x, τ ) ≡ (cid:80) x (cid:48) (cid:54) =

Original: x λ ( x, τ ; x (cid:48) ) and the transition probabilities of the embedded
Revised: x λ ( x, τ ; x (cid:48) ) and the transition probabilities of the embedded

Original: Markov chain m ( x (cid:48) | x ) .
Revised: Markov chain m ( x (cid:48) | x ) .

Original: Satisfying (cid:80) x (cid:48) (cid:54) =
Revised: Satisfying (cid:80) x (cid:48) (cid:54) =

Original: x m ( x (cid:48) | x ) = 1 , i.e., excluding self-transitions, theyform a categorical distribution over next states x (cid:48) (cid:54)
Revised: x m ( x (cid:48) | x ) = 1 , i.e., excluding self-transitions, theyform a categorical distribution over next states x (cid:48) (cid:54)

Original: =
Revised: =

Original: x .
Revised: x .

Original: Under direction-time independence, thewaiting time in each state is thus statistically independent of the jump direction.
Revised: Under direction-time independence, thewaiting time in each state is thus statistically independent of the jump direction.

Original: In this setting, for each state x , we have the waiting time measures F ( τ | x ) = 1 − Λ ( τ | x )with survival/tail functions Λ ( τ | x ) =
Revised: In this setting, for each state x , we have the waiting time measures F ( τ | x ) = 1 − Λ ( τ | x )with survival/tail functions Λ ( τ | x ) =

Original: exp (cid:0) − (cid:82) τ 0 d τ (cid:48) λ
Revised: exp (cid:0) − (cid:82) τ 0 d τ (cid:48) λ

Original: ( x, τ (cid:48) ) (cid:1) and their densities f
Revised: ( x, τ (cid:48) ) (cid:1) and their densities f

Original: ( τ | x ) =
Revised: ( τ | x ) =

Original: λ ( x, τ ) Λ ( τ | x ) .
Revised: λ ( x, τ ) Λ ( τ | x ) .

Original: For convenience of notation, we also introduce the forward transition operator ofthe embedded Markov chain M acting on a test-function g by M g ( x, t ) ≡ (cid:80) x (cid:48)
Revised: For convenience of notation, we also introduce the forward transition operator ofthe embedded Markov chain M acting on a test-function g by M g ( x, t ) ≡ (cid:80) x (cid:48)

Original: (cid:54) =
Revised: (cid:54) =

Original: x m ( x | x (cid:48) ) g ( x (cid:48) , t )and its backward adjoint M † , for which M † g ( x, t ) ≡ (cid:80) x (cid:48) (cid:54) =
Revised: x m ( x | x (cid:48) ) g ( x (cid:48) , t )and its backward adjoint M † , for which M † g ( x, t ) ≡ (cid:80) x (cid:48) (cid:54) =

Original: x m ( x (cid:48) | x ) g ( x (cid:48) , t ) holds.
Revised: x m ( x (cid:48) | x ) g ( x (cid:48) , t ) holds.

Original: Further, a homogeneous continuous-time Markov chain (CTMC) is a homogeneous and direction-timeindependent CTSMC which has constant rates in each state.
Revised: Further, a homogeneous continuous-time Markov chain (CTMC) is a homogeneous and direction-timeindependent CTSMC which has constant rates in each state.

Original: As such, the rates λ ( x, τ ) ≡ λ ( x ) areleft dependent only on the current state.
Revised: As such, the rates λ ( x, τ ) ≡ λ ( x ) areleft dependent only on the current state.

Original: Thus, the waiting time measures are strictly exponentialand so F ( τ | x ) = 1 − exp ( − λ ( x ) τ ) and f ( τ | x ) =
Revised: Thus, the waiting time measures are strictly exponentialand so F ( τ | x ) = 1 − exp ( − λ ( x ) τ ) and f ( τ | x ) =

Original: λ ( x ) exp ( − λ ( x ) τ ) .
Revised: λ ( x ) exp ( − λ ( x ) τ ) .

Original: Though signiﬁcantlyless expressive, inference in CTMC’s is by large parts available analytically [18].
Revised: Though signiﬁcantlyless expressive, inference in CTMC’s is by large parts available analytically [18].

Original: 1.2 Observation Model
Revised: 1.2 Observation Model

Original: In this work, we consider a hidden continuous-time semi-Markov model, where the latent process is a
Revised: In this work, we consider a hidden continuous-time semi-Markov model, where the latent process is a

Original: CTSMC { X ( t ) } t ≥ 0 .
Revised: CTSMC { X ( t ) } t ≥ 0 .

Original: Its state X ( t ) is observable indirectly by means of a dependent random variable
Revised: Its state X ( t ) is observable indirectly by means of a dependent random variable

Original: Y ( t ) ∈
Revised: Y ( t ) ∈

Original: Y on an observation space Y available only at discrete time instances t 0 < t 1 < · · · < t Kassuming values y 0 , y 1 , . .
Revised: Y on an observation space Y available only at discrete time instances t 0 < t 1 < · · · < t Kassuming events y 0 , y 1 , . .

Original: , y K .
Revised: , y K .

Original: The conditional Pr( Y ( t k ) ≤ y k
Revised: The conditional P ( Y ( t k ) =

Original:  | X ( t k ) = x )  is assumedto be known at observation instances.We collect all discrete event data in the measurement set E ≡ { ( y k , t k ) | k ∈ { 0 , 1 , . . . , K }} .
Revised: y k | X ( t k ) = x ) (understood as alikelihood in x ) is assumed to be known at observation instances.

Original: We also deﬁne the parametrizable subset E ( t (cid:48) , t ) ⊆ E whichextracts all samples within a speciﬁc time window E ( t (cid:48) , t ) ≡ { ( y k , t k ) | ( y k , t k ) ∈ E , t k ∈ [ t (cid:48) , t ) } . Finally, we will often write events in the form y [ t (cid:48) ,t ) denoting the joint event obtained by intersectingsingle observation events, i.e., y [ t (cid:48) ,t ) ≡ (cid:110)(cid:84) ( y k ,t k ) ∈E ( t (cid:48) ,t )  Y ( t k ) 
Revised: Finally, we will often write eventsin the form y [ t (cid:48) ,t ) denoting the joint event obtained by intersecting single observation events, i.e.,y [ t (cid:48) ,t ) ≡ (cid:84) t k ∈ [ t (cid:48) ,t ) { Y ( t k ) =

Original: [Equation]
Revised: y k } .

Original: 1.3 General Idea
Revised: 1.3 General Idea

Original: Latent state inference in Markov processes generally relies on the exploitation of their Markovproperty.
Revised: Latent state inference in Markov processes generally relies on the exploitation of their Markovproperty.

Original: It allows to describe the posterior latent state probability as a product of a forward andbackward term.
Revised: It allows to describe the posterior latent state probability as a product of a forward andbackward term.

Original: While the forward terms account for the probability of past observations at any timeinstance t , the backward terms describe the likelihood of future observations starting from t .
Revised: While the forward terms account for the probability of past observations at any timeinstance t , the backward terms describe the likelihood of future observations starting from t .

Original: In thesemi-Markov case, such a factorization is generally not possible, i.e. for a state X ( t ) =
Revised: In the semi-Markov case, such a factorization is generally not possible, i.e. for a state X ( t ) =

Original: x t
Revised: x t

Original: P
Revised: [Equation] (cid:124) (cid:123)(cid:122) (cid:125)

Original: [Equation]
Revised: [Equation]

Revised: (cid:1) (cid:124) (cid:123)(cid:122) (cid:125) .

Original: posterior (cid:54)∝ forward × backward (1)
Revised: posterior (cid:54)∝ forward × backward (1)

Original: However, as the name suggests, semi-Markov processes do not always violate the Markov property.
Revised: However, as the name suggests, semi-Markov processes do not always violate the Markov property.

Original: In particular, the process is Markov at jump instances.
Revised: In particular, the process is Markov at jump instances [13].

Original: Thus, with respect to state changes, we canagain ﬁnd a factorization similar to (1).
Revised: Thus, with respect to state changes, wecan again ﬁnd a factorization similar to (1).

Original: To be made precise later, let ∆ t denote the event of X ( t )transitioning to a state x ∈ X from a different state X ( t − ) (cid:54) =
Revised: To be made precise later, let ∆ t denote the event of X ( t )transitioning to a state x ∈ X from a different state X ( t − ) (cid:54) =

Original: x at time instance t .
Revised: x at time instance t .

Original: Then, in contrastto (1) it holds that
Revised: Then, in contrastto (1) it holds that

Original: [Equation]
Revised: [Equation]

Original: (cid:124) (cid:123)(cid:122) (cid:125) , posterior ∝ forward × backward (2) where P now takes on the form of a density instead of the mass functions in (1).
Revised: posterior ∝ forward × backward (2) where P now takes on the form of a density instead of the mass functions in (1).

Original: The term P (∆ t |y [0 ,t ) ) in (2) can be calculated under history dependence while the term P
Revised: The term P (∆ t |y [0 ,t ) ) in (2) can be calculated under history dependence while the term P

Original: ( y [ t,T ) | ∆ t ) is bydeﬁnition independent of any past trajectory up to t since it is conditioned on the state change att .
Revised: ( y [ t,T ) | ∆ t ) is bydeﬁnition independent of any past trajectory up to t since it is conditioned on the state change att .

Original: In the following, we refer to such quantities like those occuring on the right-hand side of (2) as"currents".
Revised: In the following, we refer to such quantities like those occuring on the right-hand side of (2) as"currents".

Original:  Forward and Backward Currents
Revised:  Forward and Backward Currents

Original: Probability currents - in the context of Markov chains - are known from the balance equation [19].
Revised: Probability currents - in the context of Markov chains - are known from the balance equation [19].

Original: They also occur in the description of other stochastic processes [12], CTSMC’s in particular aswell [15, 3].
Revised: They also occur in the description of other stochastic processes [12], CTSMC’s in particular aswell [15, 3].

Original: In this section, we establish a general framework of input and output probability andlikelihood currents under observations which is needed for the inference methods afterwards.
Revised: In this section, we establish a general framework of input and output probability andlikelihood currents under observations which is needed for the inference methods afterwards.

Original: 2.1 Input and Output Currents in Continuous-Time semi-Markov Chains
Revised: 2.1 Input and Output Currents in Continuous-Time semi-Markov Chains

Original: Consider again a CTSMC { X ( t ) } t ≥ 0 on a countable state space X .
Revised: Consider again a CTSMC { X ( t ) } t ≥ 0 on a countable state space X .

Original: Let a state x ∈ X of the chainand a time instance t ∈ R ≥ 0 be given.
Revised: Let a state x ∈ X of the chainand a time instance t ∈ R ≥ 0 be given.

Original: The following events form the core of the following section.
Revised: The following events form the core of the following section.

Original: Let ∆ h ( x, t ) ≡ { X ( t )(cid:54) =
Revised: Let ∇ h ( x, t ) ≡ { X ( t )

Original:  x, X ( t + h )  = x } denote the joint event of being in two different statesx and some x (cid:48) (cid:54) =
Revised: = x, X ( t + h ) (cid:54) = x } denote the joint event of being in two different statesx and some x (cid:48) (cid:54) =

Original: x at the beginning and end of a time-window of length h past t , i.e. the event ofleaving state x between t and t + h .
Revised: x at the beginning and end of a time-window of length h past t , i.e. the event ofleaving state x between t and t + h .

Original: Let in contrast further ∇ h ( x, t ) ≡ { X ( t )  =
Revised: Let in contrast further ∆ h ( x, t ) ≡ { X ( t ) (cid:54) =

Original: x, X ( t + h )(cid:54) = x }denote the joint event of being in state x (cid:48) (cid:54) =
Revised: x, X ( t + h ) = x }denote the joint event of being in state x (cid:48) (cid:54) =

Original: x at the beginning of the window and in state x at theend, i.e. entering state x between t and t + h .
Revised: x at the beginning of the window and in state x at theend, i.e. entering state x between t and t + h .

Original: Then, we can quantify the ﬂow of probability into xper unit time with φ ( x, t ) ≡ lim h → 0 1 h P (∆ h ( x, t )) .
Revised: Then, we can quantify the ﬂow of probability into xper unit time with φ ( x, t ) ≡ lim h → 0 1 h P (∆ h ( x, t )) .

Original: We call the quantity φ ( x, t ) the input currentof state x at t .
Revised: We call the quantity φ ( x, t ) the input currentof state x at t .

Original: Analogously, we deﬁne the output current by ψ ( x, t ) ≡ lim h → 0 1 h P ( ∇ h ( x, t )) .
Revised: Analogously, we deﬁne the output current by ψ ( x [Equation]

Original: In the following, we  omit function arguments iff they are the same for all functions.
Revised: In the following, we may omit function arguments if they are the same for all functions.

Original: Let the marginalprobability of being in state x at t be denoted by p ( x, t )
Revised: Let themarginal probability of being in state x at t be denoted by p ( x, t )

Original: ≡ P ( X ( t ) =
Revised: ≡ P ( X ( t ) =

Original: x ) .
Revised: x ) .

Original: Then, p is related to theinput and output currents by
Revised: Then, p isrelated to the input and output currents by

Original: [Equation]
Revised: [Equation]

Original: where I ψ ≡ ψ is the identity operator.
Revised: where I ψ ≡ ψ is the identity operator.

Original: This equation is equivalent to the forward equation inan inhomogeneous CTMC and the currents φ ( x, t ) , ψ ( x, t ) can directly be derived from it (seeappendix section A).
Revised: This equation is equivalent to the forward equation inan inhomogeneous CTMC and the currents φ ( x, t ) , ψ ( x, t ) can directly be derived from it (seeappendix section A).

Original: The input and output currents are related by the forward transition operator of the embedded Markov chain φ
Revised: The input and output currents are related by the forward transition operator of the embedded Markov chain φ

Original: = M ψ .
Revised: = M ψ .

Original: The following Markov property makes the currents important w.r.t.. (2).
Revised: The following Markov property makes the currents important w.r.t.. (2).

Original: Let x [ t,t (cid:48) )
Revised: Let x [ t,t (cid:48) )

Original: denote the trajectory of X ( t ) in the interval [ t (cid:48) , t ) .
Revised: denote the trajectory of X ( t ) in the interval [ t (cid:48) , t ) .

Original: Then,lim h → 0 P (cid:0) x [ t,T ) | ∆ h ( x, t ) , x [0 ,t ) (cid:1) =
Revised: Then,lim h → 0 P (cid:0) x [ t,T ) | ∆ h ( x, t ) , x [0 ,t ) (cid:1) =

Original: lim h → 0 P (cid:0) x
Revised: lim h → 0 P (cid:0) x

Original: [ t,T ) | ∆ h ( x, t ) (cid:1) for any x and t .
Revised: [ t,T ) | ∆ h ( x, t ) (cid:1) for any x and t .

Original: Thus, thecurrent φ ( x, t ) speciﬁes the probability  of the CTSMC having the Markov propertyper unit time at t a) b) c) d)     5 0 .0 .0 .0 .0 .1 . next to entering state x .
Revised: Thus, thecurrent φ ( x, t ) speciﬁes the probability per unit time of the CTSMC having the Markov property at tnext to entering state x .

Original: On the other hand ψ ( x, t ) speciﬁes the likelihood of being Markov at t andentering an arbitrary state x (cid:48) (cid:54) = x .
Revised: On the other hand ψ ( x, t ) speciﬁes the likelihood of being Markov at t andentering an arbitrary state x (cid:48) (cid:54) = x .

Original: An illustration of input and output currents is given in Fig.
Revised: An illustration of input and output currents is given in Fig.

Original: In this work, we talk about forward or ﬁltered currents speciﬁcally if we condition on past observations.
Revised: In this work, we talk about forward or ﬁltered currents speciﬁcally if we condition on past observations.

Original: For these, we write φ α ( x, t ) ≡ lim h → 0 1 h P (cid:0) ∆ h ( x, t )
Revised: For these, we write φ α ( x, t ) ≡ lim h → 0 1 h P (cid:0) ∆ h ( x, t )

Original: | y [0 ,t ) (cid:1) and ψ α ( x, t ) ≡lim
Revised: | y [0 ,t ) (cid:1) and ψ α ( x, t ) ≡lim

Original: h → 0 1 h P (cid:0) ∇ h ( x, t ) | y [0 ,t ) (cid:1) .
Revised: h → 0 1 h P (cid:0) ∇ h ( x, t ) | y [0 ,t ) (cid:1) .

Original: We use the notation α ( x, t ) =
Revised: We use the notation α ( x, t ) =

Original: P (cid:0) X ( t ) =
Revised: P (cid:0) X ( t ) =

Original: x | y [0 ,t ) (cid:1) for the forward marginals.
Revised: x | y [0 ,t ) (cid:1) for the a)     5 0 .0 .0 .0 .0 .1 . .b) c) d)

Revised: Hidden CTSMC

Revised: HSMM approximation forward marginals.

Original: For the forward currents and marginals (3) holds unaltered.
Revised: For the forward currents and marginals (3) holds unaltered.

Original: We complement the forward currents with backward currents φ β ( x, t ) ≡ lim h → 01 h P (cid:0) y
Revised: We complement the forward currents with backward currents φ β ( x, t ) ≡ lim h → 0 P (cid:0) y

Original: [ t,T ) | ∆ h ( x, t ) (cid:1) and ψ β ( x, t ) ≡lim h → 01 h P (cid:0) y
Revised: [ t,T ) | ∆ h ( x, t ) (cid:1) andψ β ( x, t ) ≡ lim h → 0 P (cid:0) y

Original: [ t,T ) | ∇ h ( x, t ) (cid:1) .
Revised: [ t,T ) | ∇ h ( x, t ) (cid:1) .

Original: We also assume a terminal condition to be ﬁxed at T∈ R .
Revised: We also assume a terminal condition to be ﬁxed at T .

Original: Forward and backward currents share important properties.
Revised: Forward and backward currents share important properties.

Original: Let β ( x, t ) =
Revised: Let β ( x, t ) =

Original: P (cid:0) y
Revised: P (cid:0) y

Original: [ t,T ) | X ( t ) =
Revised: [ t,T ) | X ( t ) =

Original: x (cid:1) be amarginal likelihood complementing α ( x, t ) .
Revised: x (cid:1)be a marginal likelihood complementing α ( x, t ) .

Original: Then, we have for  the both currents
Revised: Then, we have for α , β and the both currentsd

Original: [Equation]
Revised: [Equation]

Original: while the backward currents are related by ψ β = M † φ β .
Revised: while the backward currents are related by ψ β = M † φ β .

Original: Like (3), a resemblance to classical forwardand backward equations in a CTMC is not coincidental in (4).
Revised: Like (3), a resemblance to classical forwardand backward equations in a CTMC is not coincidental in (4).

Original: We have seen that knowledge of thecurrents φ α , ψ α and ψ β , φ β fully enables calculation of forward and backward terms.
Revised: We have seen that knowledge ofthe currents φ α , ψ α and ψ β , φ β fully enables calculation of forward and backward terms.

Revised: In thefollowing, we will outline the necessary equations to determine them.

Original: 2.2 Path Marginalization and  Integral Relations
Revised: 2.2 Obtaining the Forward and Backward Currents through Integral Equations

Original: Currents have a path-wise interpretation in the form of a marginalization.
Revised: The input and output currents have an interpretation as a marginalization over paths.

Original: This, and their relationto the forward and backward terms (4), enables us to determine how the observations contribute toforward and backward terms under history dependence.
Revised: This, andtheir relation to the forward and backward terms (4), enables us to determine how the observationscontribute to α and β in the history dependent CTSMC.

Original: In the following, we will use the shorthandx [ t (cid:48) ,t ) =
Revised: In the following, we will use the shorthandx [ t (cid:48) ,t ) =

Original: x to denote the event of the trajectory x
Revised: x to denote the event of the trajectory x

Original: [ t (cid:48) ,t ) being constant in X ( τ ) = x , ∀ τ ∈ [ t (cid:48) , t ) .
Revised: [ t (cid:48) ,t ) being constant in X ( τ ) = x , ∀ τ ∈ [ t (cid:48) , t ) .

Original: We begin by resolving a relationship between p and φ fundamental to CTSMC’s.
Revised: We begin by resolving a relationship between p and φ fundamental to CTSMC’s.

Original: Note, that theprobability per unit time of X ( t ) entering state x at t − τ , τ > 0 , and staying in x at least untilt , can be given by Λ ( τ | x ) φ ( x, t − τ ) =
Revised: Note, that theprobability per unit time of X ( t ) entering state x at t − τ , τ > 0 , and staying in x at least untilt , can be given by Λ ( τ | x ) φ ( x, t − τ ) =

Original: lim h → 0 1 h P (cid:0) x
Revised: lim h → 0 1 h P (cid:0) x

Original: [ t − τ,t ) = x, ∆ h ( x, t − τ ) (cid:1) .
Revised: [ t − τ,t ) = x, ∆ h ( x, t − τ ) (cid:1) .

Original: To obtainp ( x, t ) , we need to marginalize over all trajectories x
Revised: To obtainp ( x, t ) , we need to marginalize over all trajectories x

Original: [ t − τ,t ) having state x at time t .
Revised: [ t − τ,t ) having state x at time t .

Original: Because of the
Revised: Because of the

Original: Markov property at t − τ , we only need to marginalize over all constant trajectories x
Revised: Markov property at t − τ , we only need to marginalize over all constant trajectories x

Original: [0 ,t − τ )
Revised: [0 ,t − τ )

Original: = x .
Revised: = x .

Original: Thus, we are left to integrate over all τ > 0 to obtain p ( x, t ) .
Revised: Thus, we are left to integrate over all τ > 0 to obtain p ( x, t ) .

Revised: We stop at τ = t and insert an initialcondition from there.

Original: Doing this, we obtain the fundamentalrelationship
Revised: Doing this, we obtain the fundamental relationship

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: where L is a convolutional integral operator deﬁned by L φ ( x, t ) ≡ (cid:82) t 

Revised: where L is a convolutional integral operator deﬁned by L φ ( x, t ) ≡ (cid:82) t

Original: d τ Λ ( t − τ | x ) φ ( x, τ ) .
Revised: d τ Λ ( t − τ | x ) φ ( x, τ ) .

Original: The inhomogeneity g φ : X × R ≥ 

Revised: The inhomogeneity g p : X × R ≥

Original: → R ≥ 0  implements the initial condition.
Revised: → R ≥ 0 is a known general function and implements the initialcondition.

Revised: Choices for g p will be discussed in section 4.

Original: Instead of asking for x tobe kept from t − τ at least until t , we may also demand a transition shortly after t to another state.
Revised: Instead of asking for x to be kept from t − τat least until t , we may also demand a transition shortly after t to another state to obtain an expression for ψ .

Original: Thisprobability per unit time can then be given by λ ( x, τ ) Λ ( τ | x ) φ ( x, t − τ ) =
Revised: This probability per unit time can then be given by λ ( x, τ ) Λ ( τ | x ) φ ( x, t − τ ) =

Original: f ( τ ) φ ( x, t − τ ) =
Revised: f ( τ ) φ ( x, [Equation] − τ ) =

Original: lim h → 0 1 h P (cid:0) ∇ h ( x, t − τ ) , x [ t − τ,t ) 
Revised: [Equation] → [Equation] h P (cid:0) ∇ h ( x, t − τ ) , x [ t − τ,t ) =

Original: = x, ∆ h ( x, t − τ ) (cid:1) .
Revised: x, ∆ h ( x, t − τ ) (cid:1) .

Original: As the result of a marginalization overtrajectories similar to (5), i.e. integration over τ > 0 , we then obtain [Equation]
Revised: As the result of amarginalization over trajectories similar to (5), i.e. integration over τ > 0 , we then obtainψ =

Original: [Equation] with the integral operator F φ ( x, t ) ≡ (cid:82) t 0{+
+} d τ f ( t − τ | x ) φ ( x, τ ) .
Revised: F φ + g φ (6) with the integral operator F φ ( x, t ) ≡ (cid:82) t {+
+} d τ f ( t − τ | x ) φ ( x, τ ) .

Original: Again, g φ implements the initialcondition.
Revised: Again, g φ implements the initialcondition.

Original: Note, that we can understand this as an expectation ψ ( x, t ) =
Revised: Note, that we can understand this as an expectation ψ ( x, t ) =

Original: EF [ φ ( x, t − · )] over thewaiting time measure F ( · | x) .
Revised: E [ φ ( x, t − · )] over the waiting time in state x .

Original: Differentiating (5) w.r.t.
Revised: Differentiating (5) w.r.t.

Original: time t and comparing the result with (3)and (6), we get the relation of the inhomogeneities dd t g p + g φ = 0 .
Revised: time t and comparing the result with (3) and (6), weget the relation of the inhomogeneities dd t g p + g φ = 0 .

Original: Remembering that φ = M ψ , wecan obtain the two identities ψ = FM ψ + M† g ψ and φ = MF φ + g φ where g φ = M† g ψ .
Revised: Remembering that φ = M ψ , we can obtainthe two identities ψ = FM ψ + M g ψ and φ = MF φ + g φ where g φ = M g ψ .

Original: In the following, we account for observations and then close with the backward currents.
Revised: In the following, we account for observations and then close with the backward currents.

Original: Letυ ( x, t (cid:48) , t ) ≡ P (cid:0) y
Revised: Letυ ( x, t (cid:48) , t ) ≡ P (cid:0) y

Original: [ t (cid:48) ,t ) | x [ t (cid:48) ,t )
Revised: [ t (cid:48) ,t ) | x [ t (cid:48) ,t )

Original: =
Revised: =

Original: x (cid:1) /P
Revised: x (cid:1) /P

Original: (cid:0)
Revised: (cid:0)

Original: y
Revised: y

Original: [ t (cid:48) ,t )
Revised: [ t (cid:48) ,t )

Original: | y [0 ,t (cid:48) )
Revised: | y [0 ,t (cid:48) )

Original: (cid:1) be the normalized joint observationlikelihood of the constant latent trajectory x
Revised: (cid:1) be the normalized joint observationlikelihood of the constant latent trajectory x

Original: [ t (cid:48) ,t )
Revised: [ t (cid:48) ,t )

Original: =
Revised: =

Original: x .
Revised: x .

Original: The normalizer P (cid:0) y
Revised: The normalizer P (cid:0) y

Original: [ t (cid:48) ,t )
Revised: [ t (cid:48) ,t )

Original: | y [0 ,t (cid:48) )
Revised: | y [0 ,t (cid:48) )

Original: (cid:1)is the same as implicitly used when normalizing in a CTMC ﬁltering update.
Revised: (cid:1) is thesame as implicitly used when normalizing in a CTMC ﬁltering update.

Original: The υ ( x, t (cid:48) , t )is then a t -family of piece-wise constant functions with discontinuities in the t (cid:48) -direction(or
Revised: The υ ( x, t (cid:48) , t ) is then at -family of piece-wise constant functions with discontinuities in the t (cid:48) -direction

Original:  vice-versa).
Revised: (or vice-versa).

Original: For convenience, we deﬁne the modiﬁed integral operator F α φ ( x, t ) ≡168 (cid:82) t 0 d τ f υ ( τ, t | x ) φ ( x, τ ) , where f υ ( τ, t | x ) ≡ υ ( x, τ, t ) f
Revised: Forconvenience, we deﬁne the modiﬁed integral operator F α φ ( x, t ) ≡ (cid:82) t 0 d τ f υ ( τ, t | x ) φ ( x, τ ) ,where f υ ( τ, t | x ) ≡ υ ( x, τ, t ) f

Original: ( t − τ | x ) .
Revised: ( t − τ | x ) .

Original: Deﬁne L α φ ( x, t )for Λ υ ( τ, t | x ) ≡ υ ( x, τ, t ) Λ ( t − τ | x ) analoguously.
Revised: We also deﬁne L α φ ( x, t ) for Λ υ ( τ, t | x ) ≡υ ( x, τ, t ) Λ ( t − τ | x ) analoguously.

Original: Let further υ 0 ( x, t ) ≡ υ ( x, 0 , t ) be ashorthand for υ in the interval [0 , t ) .
Revised: Let further υ 0 ( x, t ) ≡ υ ( x, 0 , t ) be a shorthand for υ in theinterval [0 , t ) .

Original: Then, we can generalize (5) and (6) to obtain
Revised: Then, we can generalize (5) and (6) to obtain

Original: [Equation]
Revised: [Equation]

Original: andψ
Revised: and

Original: [Equation]
Revised: [Equation]

Original: F [Equation] From (7) immediately follow the identities ψ α = F α M ψ α + υ 0 M g ψ and φ α = MF α φ α +υ 0 g φ .
Revised: From (7) immediately follow the identities ψ α = F α M ψ α + υ 0 M g ψ and φ α = MF α φ α +υ 0 g φ .

Original: The backward currents obey relations analogous to (7).
Revised: The backward currents obey relations analogous to (7).

Original: In the following, wewill perform the similar construction.
Revised: In the following, wewill perform the similar construction.

Original: Consider the product f
Revised: Consider the product f

Original: ( τ | x ) ψ β ( x, t + τ ) =
Revised: ( τ | x ) ψ β ( x, t + τ ) =

Original: lim h → 01 h P ( y [ t,T ) , ∇ h
Revised: lim h → 0 P ( y [ t,T ) , ∇ h

Original: x, t + τ ) , x [ t,t + τ ) =
Revised: x, t + τ ) , x [ t,t + τ ) =

Original: x | ∆ h ( x, t− τ )) .
Revised: x | ∆ h ( x, t )) .

Original: It gives the probability perunit time of keeping state x in the interval [ t, t + τ ) , then leaving, and along the path encounteringany future observations, all conditioned on entering state x at t .
Revised: It gives the likelihood of keepingstate x in the interval [ t, t + τ ) , then leaving, and along the path encountering any future observations, given an entry in state x at t .

Original: Thus, φ β ( x, t ) can be obtainedagain by marginalization over future trajectories.
Revised: Thus, φ β ( x, t ) can be obtained again by marginalization overfuture trajectories.

Original: The marginal likelihood β ( x, t ) can also beshown to maintain an analogous relationship with Λ ( τ | x ) ψ β ( x, t + τ ) .
Revised: The marginal likelihood β ( x, t ) can also be shown to maintain an analogousrelationship with Λ ( τ | x ) ψ β ( x, t + τ ) .

Original: This is obtained by atwo-side marginalization along state entry ∆ h ( x, t − τ ) and exit ∇ h ( x, t + τ (cid:48) ) while implyingno information on the time of entry t − τ .
Revised: This is obtained by a two-side marginalization along stateentry ∆ h ( x, t − τ ) and exit ∇ h ( x, t + τ (cid:48) ) while implying no information on the time of entry t − τ .

Original: We refer to appendix section A for the derivation,since it takes more steps.
Revised: We refer to appendix section A for the derivation, since it takes more steps.

Original: In the reverse direction as we did for the forward case (7), we similarly normalize β and the backwards currents ψ β , φ β by P (cid:0) y
Revised: In the reverse directionas we did for the forward case (7), we similarly normalize β and the backwards currents ψ β , φ βby P (cid:0) y

Original: [ t,t (cid:48) ) | y [0 ,t ) (cid:1) .
Revised: [ t,t (cid:48) ) | y [0 ,t ) (cid:1) .

Original: This requires a fullforward pass to be performed before the backward calculations.
Revised: This requires a full forward pass to be performed before the backward calculations.

Original: Deﬁne the integral operators F β ψ ( x, t ) ≡ (cid:82) T t d τ f υ ( t, τ | x ) ψ ( x, τ ) and L β ψ ( x, t ) for Λ υ ( t, τ | x ) analogously.
Revised: Deﬁne the integral operators F β ψ ( x, t ) ≡ (cid:82) T t d τ f υ ( t, τ | x ) ψ ( x, τ ) and L β ψ ( x, t )for Λ υ ( t, τ | x ) analogously.

Original: Let further again υ T ( x, t ) ≡ υ ( x, t, T ) be a shorthand for υ in the interval [ t, T ) .
Revised: Let further again υ T ( x, t ) ≡ υ ( x, t, T ) be a shorthand for υ in theinterval [ t, T ) .

Original: As a conclusion, weobtain the relations
Revised: As a conclusion, we obtain the relationsβ =

Revised: L

Original: [Equation]
Revised: [Equation]

Original: ) where the inhomogeneities γ p and γ ψ implement the terminal condition.
Revised: ψ [Equation] + υ T γ p and φ β = F β ψ β + υ T γ ψ , (8) where the inhomogeneities γ p and γ ψ implement the terminal condition.

Original: Note, that we write γ todistinguish from the g for the initial conditions.
Revised: Note, that we write γ todistinguish from the g for the initial conditions.

Original: Again by differentiation and consultation of ( ?? ),we can ﬁnd dd t γ p + γ φ = 0 .
Revised: Again by differentiation and consultation of (4),we can ﬁnd dd t γ p + γ φ = 0 .

Original: Like before, the recurrence relations φ β = M † F β φ β + υ T γ ψ andφ β
Revised: Like before, the recurrence relations φ β = M † F β φ β + υ T γ ψ andφ β

Original: =
Revised: =

Original: F β M † φ β + υ T M † γ φ follow.
Revised: F β M † φ β + υ T M † γ φ follow.

Original: Also, γ ψ = M † γ φ .
Revised: Also, γ ψ = M † γ φ .

Original: Note, that if there are no observations and weonly keep the terminal condition, a particular case of both equations in (8) similar to (5) and (6) isobtained.
Revised: Note, that if there are no observations and weonly keep the terminal condition, a particular case of both equations in (8) similar to (5) and (6) isobtained.

Original:  Injecting Observations into Kolmogorov’s Equations
Revised:  Integro-Differential Forward and Backward Equations

Original: We ﬁrst brieﬂy sketch the derivation of Kolmogorov’s forward equation.
Revised: We ﬁrst brieﬂy sketch the derivation of Kolmogorov’s forward equation.

Original: Our goal is a direct relationbetween dd t p and p .
Revised: Our goal is a direct relationbetween dd t p and p .

Original: We start out with (3) relating dd t p and ψ .
Revised: We start out with (3) relating dd t p and ψ .

Original: We then have (6) expressing ψ in termsof φ and we also have (5) expressing p in terms of φ .
Revised: We then have (6) expressing ψ in termsof φ and we also have (5) expressing p in terms of φ .

Original: Convolutional equations can be convenientlytreated using the Laplace transform.
Revised: Convolutional equations can be convenientlytreated using the Laplace transform.

Original: We use it on (5) to express φ by a convolution of p , then plugthe result into (6) to express ψ in terms of p .
Revised: We use it on (5) to express φ by a convolution of p , then plugthe result into (6) to express ψ in terms of p .

Original: The latter result is then inserted into (3) to express dd t pin terms of p .
Revised: The latter result is then inserted into (3) to express dd t pin terms of p .

Original: The result in the time domain is then
Revised: The result in the time domain is then

Original: [Equation]
Revised: [Equation]

Original: (9) with the integral operator K p ( x, t ) ≡ (cid:82) t 0{+
+} d τ κ ( x, t − τ ) p ( x, τ ) .
Revised: [Equation] τ ) + g ( x, t ) with the integral operator K p ( x, t ) ≡ (cid:82) t {+
+} d τ κ ( x, t − τ ) p ( x, τ ) .

Original: The memory kernel κ is thengiven by L { κ } ( x, s ) ≡ ˜ f  ( s | x ) (cid:14) ˜Λ ( s | x ) with˜ f and ˜Λ theLaplace transforms of f and Λ withthe Laplace variable s . The symbol L thereby denotes the Laplace transform.
Revised: The memory kernel κ is thengiven by L { κ } ( x, s ) ≡ L { f } ( s | x ) (cid:14) L { Λ } ( s | x ) with the symbol L denoting the Laplacetransform with the Laplace variable s .

Original: The inhomogeneity gwill be discussed at the end of the section.
Revised: The inhomogeneity g will be discussed at the end of thesection.

Original: Before we consider the backward case, we implement observations.
Revised: In the following, we implement observations and introduce the backward case.

Original: Equation (9) is not simplyunderstood as a  marginalizationover paths like (5) or (6).
Revised: Equation (9) is notsimply understood as a path marginalization like (5) or (6).

Original: However,we can show that the path-wisecontribution from the observation likelihood carries over from (7) and (8) to equation (9) under thecondition that theobservation likelihood function υ ( x, t (cid:48) , t ) is piecewise constant. 
Revised: However, the path-wise contributionof the observation likelihood carries over from (7) and (8) to equation (9) if the likelihood functionυ ( x, t (cid:48) , t ) is piecewise constant. The analogue can be shown for the backward case.

Original: The rationale isto subdivide the time-domain into intervals where υ ( x, t (cid:48) , t ) is constant under variation of  t .
Revised: The rationale isto subdivide the time-domain into intervals where υ ( x, t (cid:48) , t ) is constant under variation of either tin the forward case or t (cid:48) in the backward case.

Original: Thetwo-argument function ¯ υ( x, τ ) ≡ υ ( x, τ, t ) is then a simple stepped function. Under this condition,we repeat the steps to obtain (9) but instead use our extended equations (7) under a time shift. We canthen glue the obtained sequence of equations back together to obtain [Equation]
Revised: The two-argument functions ¯ υ

Original: with the modiﬁed integral operator K α α ( x, t ) ≡ (cid:82) t 0 d τ κ υ ( τ, t, x ) p ( x, τ ) where κ υ ( x, τ, t ) ≡υ ( x, τ, t ) κ ( x, t − τ )  .
Revised: ( x, τ ) ≡ υ ( x, τ, t )(forward) and υ ( x, τ ) ≡ υ ( x, t (cid:48) , τ ) (backward) are then simple stepped functions under applicationof L .

Original: We conclude with the backward equation.
Revised: The following derivations are executed in detail in Appendix section A.

Original: This time we subdivide into timeintervals where υ ( x, t (cid:48) , t ) is constant under variation of t (cid:48) . The two-argument function υ ( x, τ ) ≡υ ( x, t (cid:48) , τ ) is then again a simple stepped function.
Revised: To obtain the forward equation, we repeat the steps to obtain (9) but instead use our extendedequations (7) for each time interval.

Original: For each time-interval we do the following steps.
Revised: For the backward equation, we do the following steps.

Original: With (4) and the two equations in (8) we have a similar toolset to the forward case.
Revised: With(4) and the two equations in (8) we have a similar toolset to the forward case.

Original: Thus, we applythe Laplace transform to the left equation of (8).
Revised: Thus, we apply the Laplace transform to the left equation of (8).

Original: We then insert ψ β expressed through β into the rightequation of (8) which gets us φ β in terms of β .
Revised: We then insert ψ β expressed through β into the rightequation of (8) which gets us φ β in terms of β .

Original: Inserting this expression into (4) yields the backwardequation valid for the considered time-interval .
Revised: Inserting this expression into (4) yields the backwardequation valid for the considered time-interval.

Original: Again glueing the obtained sequence of equationsback together, we ﬁnally obtaind
Revised: Joining the obtained sequences of equations backtogether in both cases yields the forward and backward equations [Equation] and

Original: [Equation]
Revised: [Equation]

Original: (11) with the  integral operator K β β ( x, t ) ≡ (cid:82)  T t d τ κ υ ( x, t, τ ) β ( x, τ )  .
Revised: (10) with the modiﬁed integral operators K α α ( x, t ) ≡ (cid:82) t 0 d τ κ υ ( τ, t, x ) p ( x, τ ) and K β β ( x, t ) ≡(cid:82) T t d τ κ υ ( x, t, τ ) β ( x, τ ) where κ υ ( x, t (cid:48) , t ) ≡ υ ( x, t (cid:48) , t ) κ ( x, t − t (cid:48) ) .

Original: Both inhomogeneities g and γcan be built from the inhomogeneities g p , g φ and γ p , γ ψ from section 2.2.
Revised: Both inhomogeneities gand γ can be built from the inhomogeneities g p , g φ and γ p , γ ψ from section 2.2.

Original: They satisfy the equationsg ≡ ( M − I )
Revised: They satisfy theequations g ≡ ( M − I )

Original: ( g φ − K α g p ) and γ ≡ (cid:0)
Revised: ( g φ − K α g p ) and γ ≡ (cid:0)

Original: I − M † (cid:1) ( K β γ p − γ ψ ) .
Revised: I − M † (cid:1) ( K β γ p − γ ψ ) .

Original: We will consider these in the nextsection.
Revised: We will consider these in thenext section.

Original: Finally, note that equations (10) and (11) can not simply be multiplied to obtain the full Bayesian posterior w.r.t.
Revised: Finally, note that α and β can not simply be multiplied to obtain the full Bayesianposterior w.r.t.

Original: the observations.
Revised: the observations.

Original:  Latent State Inference
Revised:  Latent State Inference

Original: There are two possibilities to realize the forward and backward algorithms.
Revised: There are two possibilities to realize the forward and backward algorithms.

Original: We exemplify this on theforward algorithm in the following.
Revised: We exemplify thison the forward algorithm in the following.

Original: In popular HSMM literature [22], the discrete-time equivalent ofthe products φ α and/or ψ α are fully calculated and α is derived from them using (3).
Revised: In popular HSMM literature [22], the discrete-timeequivalent of the products φ α and/or ψ α are fully calculated and α is derived from them using(3).

Original: This generalprocedure poses the computationally most efﬁcient overall solution under two circumstances.
Revised: This general procedure poses the computationally most efﬁcient overall solution under twocircumstances.

Original: First,if we also aim to calculate the full Bayesian posterior P (cid:0) X ( t ) = x | y [0 ,T ) (cid:1) for all t we need φ αand/or ψ α anyway.
Revised: First, if we also aim to calculate the full Bayesian posterior P (cid:0) X ( t ) = x | y [0 ,T ) (cid:1)for all t we need φ α and/or ψ α anyway.

Original: Second, if the classical memory kernel K and thus K α is hard to evaluate (e.g.by numerical inverse Laplace transform), an evaluation of F and thus F α might be computationallypreferrable, even if we need to perform an integration afterwards.
Revised: Second, if the classical memory kernel K and thus K α is hardto evaluate (e.g. by numerical inverse Laplace transform), an evaluation of F and thus F α might becomputationally preferrable, even if we need to perform an integration afterwards.

Original: In this work, we chose to evaluateα and β directly from (10)and (11) using a ﬁnite-element approach.
Revised: In this work, wechose to evaluate α and β directly from (10) using a ﬁnite-element approach.

Original: Thus, we introduce a time-mesht 0 < t 1 < · · · < t K , where we set each t k exactly at the times of observation.
Revised: Thus, we introduce atime-mesh t 0 < t 1 < · · · < t K , where we set each t k exactly at the times of observation.

Original: Boundary Conditions.
Revised: Initial and Terminal Conditions.

Original: Until now, we assumed the boundary conditions g and γ to be known butdidn’t specify them.
Revised: Until now, we assumed the boundary conditions g and γ to beknown but didn’t specify them.

Original: Compared to CTMC’s, the boundary conditions are not single states but semiinﬁnite trajectories starting .
Revised: Compared to CTMC’s, the boundary conditions are not single statesbut semi-inﬁnite trajectories.

Original: In HSMM literature [22], there are two popular boundary conditions.
Revised: In HSMM literature [22], there are two popular boundary conditions.

Original: The ﬁrst assumes, a transition occures both before t 0 and after t K .
Revised: The ﬁrst assumes, a transition occures both before t 0 and after t K .

Original: The second assumes a steadystate before t 0 and an uninformed progression after t K .
Revised: The second assumes a steadystate before t 0 and an uninformed progression after t K .

Original: The latter corresponds to the case that isclassically expressed by a vector of ones in HMM’s when the backward term is initialized.
Revised: The latter corresponds to the case that isclassically expressed by a vector of ones in HMM’s when the backward term is initialized.

Original: More onboundary conditions and their exact expression are found in appendix section A.
Revised: More onboundary conditions and their exact expression are found in appendix section A.

Original: 4.1 Unidirectional Forward and Backward Algorithms
Revised: 4.1 Unidirectional Forward and Backward Algorithms

Original: We formulate the general forward and backward algorithms in appendix section C.
Revised: We formulate the general forward and backward algorithms in appendix section C.

Original: The interpolantsα k and β k valid in the k -th time interval t ∈ [ t k , t k +1 ) obey the following integral equations
Revised: The interpolantsα k and β k valid in the k -th time interval t ∈ [ t k , t k +1 ) obey the following integral equations

Original: [Equation]
Revised: [Equation]

Original: where we introduce the truncated integral operators K [ a,b ) α k ≡ (cid:82) b
Revised: where we introduce the truncated integral operators K [ a,b ) α k ≡ (cid:82) b

Original: a d τ κ ( t − τ, x ) α k ( x, τ ) and
Revised: a d τ κ ( t − τ, x ) α k ( x, τ ) and

Original: K ( a,b ] β k ≡ (cid:82) b a d τ κ ( τ − t, x ) β k ( x, τ ) .
Revised: K ( a,b ] β k ≡ (cid:82) b a d τ κ ( τ − t, x ) β k ( x, τ ) .

Original: The inhomogeneities g k and γ k are built during runtime from the overall boundary condition g 0 , the previous interpolants α k (cid:48) , k (cid:48) <
Revised: The inhomogeneities g k and γ k are built during runtimefrom the overall boundary condition g 0 , the previous interpolants α k (cid:48) , k (cid:48) <

Original: k , and the scalingfunction υ .
Revised: k , and the scalingfunction υ .

Original: The analogue with γ K and β k (cid:48) , k (cid:48) > k holds for the backward case.
Revised: The analogue with γ K and β k (cid:48) , k (cid:48) > k holds for the backward case.

Original: 4.2 Forward-Backward Algorithm - Posterior Marginals
Revised: 4.2 Forward-Backward Algorithm - Posterior Marginals

Original: Obtaining the posterior marginals is possible on the level of currents.
Revised: Obtaining the posterior marginals is possible on the level of currents.

Original: Since they mark times when the
Revised: Since they mark times when the

Original: CTSMC has the Markov property, we can build products of the two normalized currents ψ α ( x, t ) =
Revised: CTSMC has the Markov property, we can build products of the two normalized currents ψ α ( x, t ) =

Original: lim h → 0 1 h P ( ∇ h ( x, t ) | y [0 ,t ) ) and ψ β ( x, t ) =
Revised: lim h → 0 1 h P ( ∇ h ( x, t ) | y [0 ,t ) ) and ψ β ( x, t ) =

Original: lim h → 0 1 h P ( y [ t,T ) | ∇ h ( x, t ))
Revised: lim h → 0 1 h P ( y [ t,T ) | ∇ h ( x, t ))

Original: / P ( y [ t,T ) |y [0 ,t ) ) to obtain the posterior current ˆ ψ ( x, t ) =
Revised: / P ( y [ t,T ) |y [0 ,t ) ) to obtain the posterior current ˆ ψ ( x, t ) =

Original: lim h → 0 1 h P ( ∇ h ( x, t ) |
Revised: lim h → 0 1 h P ( ∇ h ( x, t ) |

Original: y
Revised: y

Original: [0 ,T ) ) .
Revised: [0 ,T ) ) .

Original: which we can usein (3) to obtain the derivative dd t ˆ p .
Revised: which we canuse in (3) to obtain the derivative dd t ˆ p .

Original: However, we discuss this approach in appendix section C. Althoughit being preferable from a runtime perspective, the approach is more susceptible to numerical errors. A numerically preferable and more classical approach  uses a two-sided path marginalization and ispopular in discrete-time HSMM literature [22].
Revised: However, we discuss this approach in appendix section A. Although it being preferable from a runtime perspective, an more classical approach popular indiscrete-time HSMM literature [22] uses a two-sided path marginalization which allows usage ofinterpolation-based numerical methods.

Original: Thus, we suggest calculating the posterior marginalsusing [Equation]
Revised: In this work, we calculate the posterior marginals usingˆ p ( x,

Original: [Equation]
Revised: [Equation]

Original: Note that both currents φ α and ψ β are obtained as by-products without additional computation fromthe unidirectional forward and backward algorithms introduced in section 4.1.
Revised: We need to add that both currents φ α and ψ β are obtained as by-products without additional computation from the unidirectional forward and backward algorithms introduced in section 4.1.

Original: Although (13) isa two-dimensional integral and therefor scales quadratically with T , it is linear in |X| since onlycomponents of one state occur.
Revised: Although(12) is a two-dimensional integral and therefor scales quadratically with T , it is linear in |X| sinceonly components of one state occur.

Original: An algorithmic description  can be found in appendix section D.
Revised: An algorithmic description and an explanation how φ α and ψ βare obtained as by-products can be found in appendix section C.

Original: 4.3 Viterbi-type Algorithm - Latent Trajectory Point Estimation
Revised: 4.3 Viterbi-type Algorithm - Latent Trajectory Point Estimation

Original: Besides latent state marginals, point estimates of whole posterior paths are as well important in furtherinference tasks.
Revised: Besides latent state marginals, point estimates of whole posterior paths are as well important in furtherinference tasks.

Original: Originally stemming from HMM’s, the Viterbi algorithm which calculates MAP pathestimates has been successfully adopted to HSMM’s
Revised: Originally stemming from HMM’s, the Viterbi algorithm which calculates MAP pathestimates has been successfully adopted to HSMM’s

Original: However, like in hidden CTMC’s, theunknown chain length, i.e. the number of state changes until the terminal state is reached, makes thealgorithm hard to translate to continuous time.
Revised: However, like in hidden CTMC’s, theunknown chain length, i.e. the number of state changes until the terminal state is reached, makes thealgorithm hard to translate to continuous time.

Original: One is therefor often content with knowing the MAPstate only at observation times [14].
Revised: One is therefor often content with knowing the MAPstate only at observation times [14].

Original: The latter can in principle be calculated for hidden CTSMC’susing our forward equation (9).
Revised: The latter can in principle be calculated for hidden CTSMC’susing our forward equation (9).

Original: Nonetheless, for a ﬁnite number of jumps, the framework of currentsallows the formulation of a scalable full-path algorithm.
Revised: Nonetheless, for a ﬁnite number of jumps, the framework of currentsallows the formulation of a scalable full-path algorithm.

Original: A max-product-type reformulation of (x)allows an introduction of a max-input current
Revised: A max-product-type reformulation of (7)allows an introduction of a max-input current

Original: [Equation]
Revised: [Equation]

Original: which - bluntly speaking - substitutes the marginalization in φ α with a supremum.
Revised: which - bluntly speaking - substitutes the marginalization in φ α with a supremum.

Original: To keeptrack of the jump count, we equip this current with an additional parameter n .
Revised: To keeptrack of the jump count, we equip this current with an additional parameter n .

Original: We also introduce a maximizing transition operator M max for the embedded Markov chain M max g ( x, t ) ≡max x[Equation] = [Equation]
Revised: We also introduce a maximizing transition operator M max for the embedded Markov chain M max g ( x, t ) ≡max x

Original: m

Original: [Equation]
Revised: [Equation]

Original: x  | x [Equation] ) g ( x (cid:48) , t ) .
Revised: x m ( x | x (cid:48) ) g ( x (cid:48) , t ) .

Original: The transition operator M max resembles M where the sum over theset X \ { x } is exchanged by a maximization.
Revised: The transition operator M max resembles M where the sum over theset X \ { x } is exchanged by a maximization.

Original: The constituting recursion for φ max ( x, t ; n ) is thengiven by
Revised: The constituting recursion for φ max ( x, t ; n ) is thengiven by

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: which calculates the max-currents φ max ( x, t ; n + 1) associated with the incremented chain lengthn + 1 .
Revised: which calculates the max-currents φ max ( x, t ; n + 1) associated with the incremented chain lengthn + 1 .

Original: A detailled descriptionand a comparison to the original Viterbi algorithm is found in appendixsection A. The MAP terminal state x is related to the currents by
Revised: A detailed description is found in appendix section A. The MAP terminal state x is related tothe currents by

Original: [Equation]
Revised: [Equation]

Original: Let us ignore the chain length marginal P ( n ) for now and assume we found a maximizing tuple( x ∗ , τ ∗ , n ∗ ) .
Revised: Let us ignore the chain length marginal P ( n ) for now and assume we found a maximizing tuple( x ∗ , τ ∗ , n ∗ ) .

Original: Then, if n ∗ > 0 , we can insert φ max ( x ∗ , τ ∗ ; n ∗ ) in (4.3) and proceed to trace backstates and transition instances until the jump counter reaches zero.
Revised: Then, if n ∗ > 0 , we can insert φ max ( x ∗ , τ ∗ ; n ∗ ) in (13) and proceed to trace backstates and transition instances until the jump counter reaches zero.

Original: However, to ﬁnd the tuple( x ∗ , τ ∗ , n ∗ ) , we need knowledge of at least some scaling factor L ( n ) ∝ P ( n ) .
Revised: However, to ﬁnd the tuple( x ∗ , τ ∗ , n ∗ ) , we need knowledge of at least some scaling factor L ( n ) ∝ P ( n ) .

Original: Proper normalization to get P ( n ) requires knowledge of n up to inﬁnity, which is intractable.
Revised: Proper normalization to get P ( n ) requires knowledge of n up to inﬁnity, which is intractable.

Original: However, a scalingfactor L ( n ) can be found recursively.
Revised: However, a scalingfactor L ( n ) can be found recursively.

Original: To do this, two more sets of input currents φ n ( x, t ) ≡ lim h → 0 1 h P (cid:0) ∆ h ( x, t ) , y [0 ,t ) , n (cid:1) and marginals p n ( x, T ) ≡ P (cid:0) X ( T ) 
Revised: To do this, two more sets of input currents φ n ( x, t ) ≡lim h → 0 1 h P (cid:0) ∆ h ( x, t ) , y [0 ,t ) , n (cid:1) and marginals p n ( x, T ) ≡ P (cid:0) X ( T ) =

Original: = x, y [0 ,t ) , n (cid:1) are introduced.
Revised: x, y [0 ,t ) , n (cid:1) are introduced.

Original: The φ n ( x, t ) and p n ( x, T ) are then updated by
Revised: The φ n ( x, t ) and p n ( x, T ) are then updated by

Original: [Equation]
Revised: [Equation]

Revised: n and

Original: [Equation]
Revised: [Equation]

Original: The currents are initialized by φ 0 ( x, t ) =
Revised: The currents are initialized by φ 0 ( x, t ) =

Original: υ (0 , t ) g φ ( x, t ) .
Revised: υ ( x, 0 , t ) g φ ( x, t ) .

Original: Note that (14) does not converge tothe ﬁxed-point φ α
Revised: Note that the left equation in (14)does not converge to the ﬁxed-point φ α

Original: but instead will vanish for n → ∞ .
Revised: but instead will vanish for n → ∞ .

Original: The factors L ( n ) ≡ P (cid:0) y [0 ,t ) , n (cid:1) =
Revised: The factors L ( n ) ≡ P (cid:0) y [0 ,t ) , n (cid:1) =

Original: 294 (cid:80) x p n ( x, T ) can then be obtained by marginalization.
Revised: (cid:80) x p n ( x, T ) can then be obtained by marginalization.

Original: Again, we refer to appendix section A. Unfortunately, to obtain an exact result, we still need to evaluate all equations for arbitrarily large n .
Revised: Again, we refer to appendixsection A. Unfortunately, to obtain an exact result, we still need to evaluate all equations for arbitrarilylarge n .

Original: For most practical scenarios, however, the most likely number of transitions n ∗ might be manageable.
Revised: For most practical scenarios, however, the most likely number of transitions n ∗ might bemanageable.

Original: We stress that the bottleneck of our algorithm is the number of transitions, not their temporal closeness.
Revised: We stress that the bottleneck of our algorithm is the number of transitions, not theirtemporal closeness.

Original:  Adaptive Step-Size HSMM
Revised:  Adaptive Step-Size HSMM

Original: One of the major advantages of continuous-time formulations is the availability of adaptive discretization [8, 4].
Revised: One of the major advantages of continuous-time formulations is the availability of adaptive discretization [8, 4].

Original: Although many continuous problems eventually need to be solved numerically on adiscrete grid, adaptive step size methods can bound the discretization error and allow a reduction ofthe amount of function evaluations.
Revised: Although many continuous problems eventually need to be solved numerically on adiscrete grid, adaptive step size methods can bound the discretization error and allow a reduction ofthe amount of function evaluations.

Original: While much more elaborate techniques with tight error boundsare available for integral equations [10], we brieﬂy sketch a simple adaptive discretization strategy toclose with a practical perspective on the contributions in this work.
Revised: While much more elaborate techniques with tight error boundsare available for integral equations [10], we brieﬂy sketch a simple adaptive discretization strategy toclose with a practical perspective on the contributions in this work.

Original: This strategy consists of maintaining a time-dependent set of samples of approximate currents andtheir timestamps Φ α ( t ) ≡ { ( φ α,k , t k ) | t K − t 0 < ∆ max , t 0 < · · · < t K , k = 0 , . .
Revised: This strategy consists of maintaining a time-dependent set of samples of approximate currents andtheir timestamps Φ α ( t ) ≡ { ( φ α,k , t k ) | t K − t 0 < ∆ max , t 0 < · · · < t K , k = 0 , . .

Original: . , K } .
Revised: . , K } .

Original: Ananalogue set is built for ψ β .
Revised: Ananalogue set is built for ψ β .

Original: Then, by inserting polynomial interpolants (e.g. afﬁne combinationsof adjacent ( φ α,k , t k ) and ( φ α,k +1 , t k +1 ) ), we can reformulate (7) so that it only contains the φ α,k , 309 t k and deﬁnite integrals of the f ( · | x ) .
Revised: Then, by inserting polynomial interpolants (e.g. afﬁne combinationsof adjacent ( φ α,k , t k ) and ( φ α,k +1 , t k +1 ) ), we can reformulate (7) so that it only contains the φ α,k , 307 t k and deﬁnite integrals of the f ( · | x ) .

Original: For these integrals, look-up tables are easily generated forspeedup.
Revised: For these integrals, look-up tables are easily generated forspeedup.

Original: The resulting linear system can then be solved to calculate the next step φ (1)
Revised: The resulting linear system can then be solved to calculate the next step φ (1)

Original: α ( x, t K + h ) .
Revised: α ( x, t K + h ) .

Original: As a commonly adapted update rule for step size h , we use half step checks φ (0 . 5)
Revised: As a commonly adapted update rule for step size h , we use half step checks φ (0 . 5)

Original: α ( x, t K + h ) , wherewe do two steps using h 2 as step size and then calculate E ≡ | φ (1)
Revised: α ( x, t K + h ) , wherewe do two steps using h 2 as step size and then calculate E ≡ | φ (1)

Original: α ( x, t K + h ) − φ (0 . 5)
Revised: α ( x, t K + h ) − φ (0 . 5)

Original: α ( x, t K + h ) | .
Revised: α ( x, t K + h ) | .

Original: The integration error E is then used in the update h → max { min { hs, h max , t (cid:48) − t K − h } , h min }with the scaling s = max { min { tol E , s max } , s min } for some pre-speciﬁed h min , h max , s min , s maxand the next observation time t (cid:48) .
Revised: The integration error E is then used in the update h → max { min { hs, h max , t (cid:48) − t K − h } , h min }with the scaling s = max { min { tol E , s max } , s min } for some pre-speciﬁed h min , h max , s min , s maxand the next observation time t (cid:48) .

Original: The resulting HSMM is then only dependent on time-differemces, i.e. still homogeneous, but adaptsits step-size.
Revised: The resulting HSMM is then only dependent on time-differences, i.e. still homogeneous, but adaptsits step-size.

Original: In appendix section C, we provide the resulting update equation if afﬁne combinationsare used as interpolants between the points ( φ α,k , t k ) and an implicit Euler step rule is chosen.
Revised: In appendix section C, we provide the resulting update equation if afﬁne combinationsare used as interpolants between the points ( φ α,k , t k ) and an implicit Euler step rule is chosen.

Original:  Evaluation
Revised:  Evaluation

Original:  Our algorithms demand the solution of integral equations of the second kind with discontinuities.
Revised: Calculation of Integral Equations. Our algorithms demand the solution of integral equations of thesecond kind.

Original: For each time interval [ t, t (cid:48) ) between observations, we used abasic backward Euler method tosolve the possibly stiff equations  (7),(10), (8) or (11) with a ﬁxed number of steps calculated by
Revised: For each time interval [ t, t (cid:48) ) between observations, we used a backward Euler methodto solve the possibly stiff equations in (7), (8) or (10) with a ﬁxed number of steps calculated by

Original: N ≡ (cid:98) t (cid:48)
Revised: N ≡ (cid:98) t (cid:48)

Original: − t h (cid:99) + 1 .
Revised: − t h (cid:99) + 1 .

Original: Thus, all steps are of equal length with exception of those at the end of an intervalwhich scales dependent on t (cid:48) .
Revised: Then, all steps are of equal length h except the one at the end of the interval whichscales dependent on t (cid:48) .

Revised: The integration involved in each step is performed by interpolation betweenthe steps, where the trapezoidal rule has been used.

Revised: This method is outlined in detail in Appendixsection B.

Original: An exception is the evaluation of the adaptive step-size HSMM.
Revised: An exception is the evaluation of the adaptive step-size HSMM.

Original: There,as described in section 5, the step size is initialized by the smallest value 10 − 4 and then the adaptiveprocedure generates any further grid instances.
Revised: As described in section 5,there the step size is initialized by a smallest value 10 − 4 and then the adaptive procedure generatesany further grid positions.

Original: The integration itself is performed by interpolationbetween the steps, where the trapezoidal rule has been used.
Revised: The integration at each step is also more precise. The details are outlinedin

Original:  For forward an backward passes, eachinterval is then calculated consecutively while the inhomogeneities g and γ are updated upon enteringeach new interval.
Revised: Appendix B. For forward and backward passes, each interval is alculated consecutively while theinhomogeneities g and γ are updated upon entering each new interval as outline in the algorithmsection in Appendix C.

Original: The calculated functions α , β , ˆ p and the currents were generated using cubicspline interpolation. The exact procedure is detailled in appendix section D.
Revised: The resulting continuous functions α , β , ˆ p and the currents are then generatedfor each interval using cubic spline interpolation on the calculated steps.

Original:  We calculated several randomly generated hidden CTSMC’s specifying the set of states.
Revised: Experiments. We calculated several randomly generated hidden CTSMC’s.

Original: Waitingtimes where drawn  randomly  from a given family of distributions and transition operators of theembedded Markov chains were sampled as sets of random categorical distributions on the state space.      30 0 .0 .a) b)     5    5 c) d)     10         8 10 12 14 0.
Revised: The number of stateshas been given, but waiting time distributions and transition probabilities were drawn randomly. Foreach state, we drew a randomly parametrized waiting time distribution from a speciﬁed family ofdistributions (we allowed Gamma and Weibull).

Original: 1.2 1e For each waiting time family, we had a Gamma prior over each parameter with per-family calibratedhyperparameters.
Revised: The random parameters, were drawn from Gammadistributions with per-parameter calibrated hyperparameters.

Original: The categorical distributions were sampled drawing a sequence of iid multinomialoutcomes of which ensemble frequencies were calculated to generate a categorical mass function.
Revised: The transition probabilities of theembedded Markov chains were sampled as sets of random categorical distributions while prohibitingself-transitions.

Original: The observation times were drawn from a Gamma process with hyperparameters shape and rate,while the observation values were noisy state observations Y ( t ) N ( X ( t ) , σ ) with the std σ a givenhyperparameter.
Revised: The categorical distributions were sampled by ﬁrst drawing a vector uniformly fromthe |X| -hypercube and then projecting it to the ( |X| − 1) -simplex via normalization. The observationtimes were drawn from a point process with Gamma inter-arrival times with ﬁxed shape and rate

Original: Details are found in appendix section E.{+1.2 d) c) b) a) 10    10            0 2 4 6 8 10 
+}
Revised: {+1.2 d) c) b) a) 10    10            0 2 4 6 8 10 
+}

Original: The calculated α , β , ˆ p were compared to those calculated by a HSMM.
Revised: The calculated α , β , ˆ p were compared to those calculated by a HSMM.

Original: With a step-size of h HSMM =10 − 4 and h CTSMC = 10 − 3 , no difference was visible anymore on any calculated trajectory.
Revised: With a step-size of h HSMM =10 − 4 and h CTSMC = 10 − 3 (backward Euler step-size), no difference was visible anymore on anycalculated trajectory.

Original: In 
Revised: In Fig.

Revised: 2a, we show an exemplary progression of the forward input and backwardoutput currents.

Original:  Fig.2.b
Revised: In Fig.

Original:  the smoothed marginals ˆ p of a four-state CTSMC are compared to those obtained from itssteady-state CTMC approximation [13].
Revised: 2b the smoothed marginals ˆ p of a four-state CTSMC are compared to thoseobtained from its steady-state CTMC approximation [13].

Revised: For this, smoothed mean and varianceof b ( X ( t )) are shown.

Revised: The full trajectory of b ( X ( t )) by the original CTSMC which generated theobservations is shown as a ground truth.

Original: We built the adative step-size HSMM using our proposedmethod and ran it over the sampled trajectories as well.
Revised: We built the adaptive step-size HSMM using our proposedmethod and ran it over the sampled trajectories as well.

Original: While quickly enlarging its step-size, itremained within the margins of a total error of 10 − 3 compared to a ﬁne-grained HSMM althoughnot guaranteed by the method.
Revised: While quickly enlarging its step-size, itremained within the margins of a total error of 10 − 3 compared to a ﬁne-grained HSMM, althoughthis is not guaranteed by the method.

Original: An example run is shown in Fig. 2.c.
Revised: An example run is shown in Fig. 2c.

Original: Finally, we testet ourViterbi
Revised: Finally, we tested our

Original:  MAP estimator on the trajectories.
Revised: Viterbi MAP estimator on the trajectories.

Original: An example run is shown in 2.d.
Revised: An example run is shown in 2d.

Original: The chain length scaling factor L ( n ) ∝ P ( n ) had a comparably sharp peak in all trajectories (cf.
Revised: The chain length scalingfactor L ( n ) ∝ P ( n ) had a comparably sharp peak in all trajectories.

Original: appendix section D).

Original: We assume,this came from the observations greatly narrowing down the amount of candidate transition counts.
Revised: We assume, this came from theobservations greatly narrowing down the amount of candidate transition counts.

Original:  Conclusion
Revised:  Conclusion

Original: We have presented a forward-backward framework applicable to hidden CTSMC’s.
Revised: We have presented a forward-backward framework applicable to hidden CTSMC’s.

Original: Avoidîng statespace augmentation to enforce the Markov property, it gives rise to integral and integro-differentialforward and backward equations and a scalable Viterbi algorithm.
Revised: Avoidîng statespace augmentation to enforce the Markov property, it gives rise to integral and integro-differentialforward and backward equations and a scalable Viterbi algorithm.

Original: Adaptive step-size HSMM’scan be derived, overcoming discrete-time core problems: out-of-grid observations and steady-stateover-computation.
Revised: Adaptive step-size HSMM’scan be derived, overcoming discrete-time core problems: out-of-grid observations and steady-stateover-computation.

Original: We didn’t consider CTSMC’s not satisfying direction-time independence whichare especially interesting for coarse-graining of large Markovian systems.
Revised: We didn’t consider CTSMC’s not satisfying direction-time independence whichare especially interesting for coarse-graining of large Markovian systems.

Original: Also, speciﬁc classes ofwaiting times could allow efﬁcient computation of currents, e.g. power-law waiting times related tofractional calculus being of recent interest in physics.
Revised: Also, speciﬁc classes ofwaiting times could allow efﬁcient computation of currents, e.g. power-law waiting times related tofractional calculus being of recent interest in physics.

Original: Possible solutions would then ﬁt naturally inour framework which handles observation likelihood of past events.
Revised: Possible solutions would then ﬁt naturally in

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .
Revised: [Yes] See Section ?? .

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data areproprietary.•
Revised: [No] The code and the data areproprietary.•

Original: Did you include the license to the code and datasets?
Revised: Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A](d)
Revised: [N/A]

Original:  Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]2.
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes](b) Did you include complete proofs of all theoretical results?

Original: [Yes] In Appendix A3.
Revised: [Yes] In Appendix A3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: In theSupplement
Revised: In theSupplement

Original: (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes]
Revised: [Yes]

Original: (c) Did you report error bars (e
Revised: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?
Revised: g., with respect to the random seed after running experi- ments multiple times)?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you include the total amount of compute and the type of resources used (e
Revised: (d) Did you include the total amount of compute and the type of resources used (e

Original: g., typeof GPUs, internal cluster, or cloud provider)?
Revised: g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] In Appendix E4.
Revised: [Yes] In Appendix E

Original: If you are using existing assets (e
Revised: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...
Revised: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you mention the license of the assets?
Revised: b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c ) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d ) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e ) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]