(d) Sup pose the walker starts in vertex B. What is the probability that the wal

Question

(d) Sup pose the walker starts in vertex B. What is the probability that the walker reaches A before the walker reaches C? steps until the walker reaches A? 1.9 Consider the Markov chain with state space (1,2,3, 4,5) and matrix (e) Again assume the walker starts in C. What is the expected number of 1[0 1/32/3 0 0 2 0 0 0 1/43/4 P=3100 01/21/2 5 1 0 0 0 0 (a) Is the chain irreducible? (b) What is the period of the chain? (c) What are p1,000(2, 1),Ph,000 (2, 2),P1,000 (2, 4) (approximately)? (d) Let T be the first return time to the state 1, starting at state 1. What is the distribution of T and what is E (T)? What does this say, without any further calculation, about (1)? (e) Find the invariant probability . Use this to find the expected return time to state 2, starting in state 2.

Explanation / Answer

(a) A chain is irreducible if it has:-

Infinitely many of the pn’s need to be positive. If only finitely many are positive, and pm is the last positive one, then 0 does not communicate with m+1. If infinitely many are positive, to show that m communicates with n, let l > n be such that pl > 0. Then p(l+mn+1)(m, n) pl > 0.

Only then we can assume that the chain is irreducible

Here n=1,2,3,4,5 we have that P(X1=n|X0=1)>0 and P(X1=1|X0=n)>0 Hence its irreducible

(b) A markov chain is periodic if :-

(i) any one of the diagonal elements of the one-step transition probability matrix is positive, then that state is aperiodic. Since, periodicity is a class property, and the chain being irreducible, all the states have the same period.

(ii) If none of the diagonal elements of one-step transition probability matrix is positive, similar to the question we need to compute higher-order transition probability matrices in order to know the times n>1 for which p(n)kk>0,kS, where k is an arbitrary state in the state space S of the Markov chain. For some n>1, if the computed higher-order transition matrix becomes regular, the chain becomes aperiodic

(iii) 1 = 1/3 2 + 2/3 3

(iv) If i is recurrent, then Ni(t) is a renewal process, since the Markov property gives independence of inter arrival times XAn = Sn Sn1.
Let µii = E[XA1], the expected return time for i, we then have the following from renewal theory:

Related Questions

Navigate

• Browse (All)
• Browse #
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.