Suppose you have a continuous time Markov chain with three states, {1,2,3}. The

Question

Suppose you have a continuous time Markov chain with three states, {1,2,3}. The process stays in state i for an exponentially distributed time with rate mu_i, where mu = [1, 2, 3]. The transition matrix (for the "embedded" Markov chain) is P = [0, 2/3, 1/3; 1/3, 0, 2/3; 1/2, 1/2, 0]. The process starts in state 1, i.e., X(0) = 1.

(a) Simulate the process to find the distribution of X(1) and X(10), i.e., Find P(X(1)=i), i=1,2,3 and P(X(10)=i), i=1,2,3.

(b) Write down the infinitessimal generator, and find the probabilities exactly.

Explanation / Answer

(a) P(X(1)=1) = 0

P(X(1)=2) = 2/3

P(X(1)=3) = 1/3

P(X(10)) = [ 1 0 0 ] * P^10 where P is given probability matrix

P^10 = [ 0.291 , 0.367 , 0.342 ; 0.293 , 0.365 , 0.342 ; 0.293 , 0.367 , 0.340 ]

P(X(10)) = [0.291 , 0.367 , 0.342 ]

P(X(10)=1)= 0.291

P(X(10)=2) = 0.367

P(X(10)=3) = 0.342

(b) infinitesimal generator is transition rate mate

infinitesimal generator = [ -(2/3 +1/3) , 2/3 , 1/3 ; 1/3 , -(1/3 + 2/3) , 1/3 ; 1/2 , 1/2 , -(1/2 + 1/2) ]

=[ -1 , 2/3 , 1/3 ; 1/3 , -1 , 1/3 ; 1/2 , 1/2 , -1 ]

Related Questions

Navigate

• Browse (All)
• Browse S
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.