Consider the random walk on the integers {O, 1, 2, 3} which takes steps +1 (to t

Question

Consider the random walk on the integers {O, 1, 2, 3} which takes steps +1 (to the right) with probability 1/3 and -1 (to the left) with probability 2/3, except at the endpoints where there is reflection; this means that a step from 1 to 0 is always followed by a step from 0 to 1, and a step from 2 to 3 is always followed by a step from 3 to 2. Determine the transition matrix for this Markov chain. Calculate the stationary distribution of this Markov chain. Suppose the Markov chain has been running for a long time. What fraction of time has it spent in state 0

Explanation / Answer

The transistion matrix is

The state {0} is closed communicating class and its fraction of time is 4(2/3)=8/3=2.667(in right direction) and 4(1/3)=4/3=1.33( in left direction).

0 1 2 3 0 0 1/3 0 2/3 1 2/3 0 1/3 0 2 0 2/3 0 1/3 3 1/3 0 2/3 0

Related Questions

Navigate

• Browse (All)
• Browse C
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.