Talk:Continuous-time Markov chain

message: The example given in the article, describing a three-state space consisting of {Bull Market, Bear Market, Recession}, seems rather strange. In their ordinary meaning, the "Recession" state is not mutually distinct from the other two. It is certainly possible to have a Recession during a Bull Market or Bear Market. It would be better to define the three-state space as {Bull Market, Bear Market, Stagnant Market} or {Bull Market, Bear Market, Mixed Market}. —BarrelProof (talk) 20:03, 30 May 2013 (UTC)
replies: text: I would make that change myself, but it requires re-drawing the figure, and I don't have an easy way to do that. Incidentally, the figure also violates the MOS:NUM guideline that says "Numbers between −1 and +1 require a leading zero (0.02, not .02)". —BarrelProof (talk) 20:12, 30 May 2013 (UTC)
replies: text: Thanks for these suggestions, I've now updated the image,  Gareth Jones (talk) 21:53, 30 May 2013 (UTC)
replies: text: It looks good to me! Thanks. —BarrelProof (talk) 22:29, 30 May 2013 (UTC)
replies: text: I just noticed that there's a similar example for the discrete-time case in the Markov chain article. —BarrelProof (talk) 22:33, 30 May 2013 (UTC)
replies: text: Well spotted, I've updated that too. Gareth Jones (talk) 23:10, 30 May 2013 (UTC)
thread_title:  Strange example: {Bull Market, Bear Market, Recession} 

message: In the section about Embedded Markov chains, there seems to be a discrepancy between the definition given in terms of s_{ij} and S = - D_Q^{-1}Q. In the first definition, the diagonal values of S are given as zero, but the second definition will have them as 1.  I believe the first definition is correct, so we want instead that S = - D_Q^{-1}Q - I, which also makes more sense in the following lines when we try to find the kernel of (S-I) for the steady states of the discrete time Markov process. If S is the transition probability matrix of the EMC, we should be trying to find the stationary states of S, not S-I. I have gone ahead and made this change.76.120.32.59 (talk) 21:02, 3 June 2013 (UTC)
replies: text: You're right about getting zeroes being correct, but need S = I - \left( \operatorname{diag}(Q) \right)^{-1} Q to get them. I'll change this now. Gareth Jones (talk) 11:59, 4 June 2013 (UTC)
thread_title:  Discrepancy correction 

message: Can anyone shed light on the history of CTMCs? Richard Weber's Markov Chain course notes contain a section titled "Historical notes" (p.52) where a quote from Youschkevitch suggests Markov's own work was limited to the discrete case ("For him the only real examples of the chains were literary texts, where the two states denoted the vowels and consonants"), but I haven't made any further progress. Gareth Jones (talk) 11:06, 10 June 2013 (UTC)
thread_title:  History 

message: It is written that
:P(t) = e^{tQ}
has elements of the form
:p_{ij}(t) = \delta_{ij} + \sum_{k=1}^\infty \frac{t^k q^k_{ij}}{k!}
which equals
:p_{ij}(t) = \delta_{ij} + e^{tq_{ij}} - 1.
By definition of matrix exponential, however:
:P(t) = \sum_{k=0}^\infty \frac{t^k Q^k}{k!}.
And there is no simple elementwise expression, which is the source of complexity of calculating these. So it seems wrong... bungalo (talk) 14:10, 21 July 2013 (UTC)
thread_title:  Formula for P(t) seems incorrect 

message: > Each non-diagonal value can be computed as the product of the original state's holding time with the probability from the jump chain of moving to the given state.
This looks wrong to me. In the example, 3 is the product of 6 and 1/2, with 6 being the rate rather the holding time (which is a random variable with a mean of 1/rate). 92.110.219.57 (talk) 19:59, 6 September 2022 (UTC)
replies: text: You are correct. It should also be the expected holding time, rather than the holding time (which is a random variable). I've just fixed this. Malparti (talk) 12:20, 8 September 2022 (UTC)
thread_title:  Rate instead of holding time.