No sheets, notes, or books are allowed. The starred problem is for 855 students only. The double starred problem is for 455 students only.
Partial credit is given.
You have 90 minutes. Argue that for , . Deduce that probability that the chain ever visits state 4 starting in 3, is . Determine the closed, irreducible classes of the state space S . Suppose a certain illness can be in one of 2 stages.