A two-state Markov chain can he in one of two states: good (0) or bad (1). Assum

Question

A two-state Markov chain can he in one of two states: good (0) or bad (1). Assume that the state of the process changes from one day to the next according to the following probabilities: If it is in "good" state today, then it will be in "good" state tomorrow with probability 0.4; If it is in "bad" state today, then it will be in "bad" state tomorrow with probability 0.8. (a) Write the one-step transition matrix P. (b) Suppose it is "good" on Thursday (today), find the probability that it will be "good" on Friday and Saturday and "bad" on Sunday. (c) If it is "bad" on Thursday (today), what is the probability that it will be "bad" on Saturday? (d) In the long run, what fraction of time is the process in the "good" state?

Explanation / Answer

Suppose we donote state for current day as X0 and for next day as X1.

Also, denote "good" state by number 1 and "bad" state by number 0.

Then, given:

P(X1 = 1 | X0 = 1) = 0.4

P(X1 = 0 | X0 = 0) = 0.8

Hence, P(X1 = 0 | X0 = 1) = 1- P(X1 = 1 | X0 = 1) = 1-0.4 = 0.6

and

P(X1 = 1 | X0 = 0) = 1- P(X1 = 0 | X0 = 0) = 1-0.8 = 0.2

So, Transition matrix would be P is:

(b)

P(Good on Fri & Sat and Bad on Sun | Good on Thu)

= P(X1=1,X2=1,X3=0|X0=1)

= P(X2=1,X3=0|X1=1,X0=1) * P(X1=1|X0=1)

= P(X3=0|X2=1,X1=1,X0=1) * P(X2=1|X1=1,X0=1) * P(X1=1|X0=1)

= P(X3=0|X2=1) * P(X2=1|X1=1) * P(X1=1|X0=1)

Using Transition Matrix above for 2 consecutive days, this value is:

= P(Bad Tomorrow|Good Today) * P(Good Tomorrow|Good Today) * P(Good Tomorrow|Good Today)

= (1-0.4)*(0.4)*(0.4)

= 0.6 * 0.16

= 0.096

(c) P(Bad on Sat | Bad on Thu)

= P(X2 = 0 | X0=0)

= P(X2=0,X1=0|X0=0) + P(X2=0,X1=1|X0=0)

= P(X2=0| X1=0, X0=0) * P(X1=0|X0=0) + P(X2=0| X1=1, X0=0) * P(X1=1|X0=0)

= P(X2=0| X1=0) * P(X1=0|X0=0) + P(X2=0| X1=1) * P(X1=1|X0=0)

= P(Bad Tomorrow | Bad Today) * P(Bad Tomorrow | Bad Today) + P(Bad Tomorrow | Good Today) * P(Good Tomorrow | Bad Today)

= 0.8 * 0.8 + (1-0.4) * (1-0.8)

= 0.64 + 0.12

= 0.76

(d)

Suppose, x is the proportion of time in "Good" state in long run.

The, (1-x) is the proportion of time in "Bad" state in long run

Then, using rule for limiting distribtion,

vector (x,1-x) * transition matrix P = vector(x,1-x)

This gives:

x * 0.4 + (1-x) * 0.2 = x and x * 0.6 + (1-x) * 0.8 = 1-x

Which gives:

(1-x) * 0.2 = 0.6 * x and 0.6*x = 0.2*(1-x)

Giving: 1-x = 0.6/0.2 * x = 3*x

Hence, 4*x = 1 giving: x = 1/4 = 0.25

P(for tomorrow based on today) Good Tomorrow(X1=1) Bad Tomorrow(X1=1) Good Today(X0=1) 0.4 0.6 Bad Today(X0=0) 0.2 0.8

Related Questions

Navigate

• Browse (All)
• Browse A
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.