This is a stochastic process markov chain problem. Just need help with numbers 2

ID: 3204419 • Letter: T

Question

This is a stochastic process markov chain problem.

Just need help with numbers 2) and 3). 1) is used for the calculation for 3b which is why it is included. Thank you!

The transcribed handwriting for the questions is written below, the picture is only there for the transition matrix, and notation.

1) a) FInd all closed and irreducible sets of states, and deternine which states are recurrent and which states are transient justify.

b) Given initial probabilities, P(X0=0)=1/4, P(X0=1)=1/4, and P(X0=2) =1/2, Find i) P(Xi=2) ii) P(X1=2, X2=0) iii) P^2(0,2).

2) Derive an expression for P{X2=x2, X4=x4| X1=x1} involving a one-step and/or a two-step transition function.

(The answer is P^2(x2,x4)P(x1,x2) I am just unclear how to get there)

3) a) Show Px{Ty=n}, the hitting time, is less than or equal to P^n(x,y), the n-step transition function,

b) Show by example (using 1) that P^n(x,y), the n-step transition function, does not equal Px(Ty=n), the hitting time.

(407 t.k. 4 2fXi0( 30, //Im eseaen2,6ter y,rn)2%-.) /k .

2. P{X2=x2, X4=x4| X1=x1 } = P{X2=x2|X1=x1} P{X4=x4| X2=x2 ,X1=x1}

= P(X2,X1) P{X4=x4 | X2=x2 } [since the process is a markov chain]

= P21 P42

By Chapman-Kolmogorov's equation pn=pmpn-m

hence P3 = P2(X2,X4) P3-2(X1,X2) ,here n=3,m=2

= P2(X2,X4) P(X1,X2) (answer)

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.