PROBLEM 4.9.2 Consider a 6-state Markov chain with the following transition prob
ID: 3073337 • Letter: P
Question
PROBLEM 4.9.2 Consider a 6-state Markov chain with the following transition probability matrix: states 0 2 3 4 5 0 0.5 0.5 0 000 1 0.5 0.5 0 0 0 0 P20 0 0.4 0.3 0.2 0.1 3 0.1 0.1 0.2 0 0.5 0.1 4 0.40 0.3 0.2 0 0.1 The recurrent classes are 0l) and {5). The remaining cass 2,34) is transient. If we start in one of the transient states, we will eventually transition to one of the recurrent classes and never leave. We are interested in determining which recurrent class we will transition to when we start in one of the transient states. Insert a relay state called 5* which we will pass through only on our way to state 5. With the addition of this new state, the states 2.3.4 and 5* are transient. tss we will transition to a. b. c. d. e. Give the transition probability matrix PT between the 4 transient states. Find the corresponding S matrix of expected visits to the transient states. If we start in state 2, what is the probability we ever visit state 5? If we start in state 2, what is the probability we ever visit state 1? If we start in state 2, what is the chance we ever visit state 4?Explanation / Answer
a. The states 1, 2, 3, 4 are persistent as it is possible to return to them after leaving them while states 5 and 6 are transient as once we move from them to states 1-4, it is impossible to return.
b. To obtain the stationary distribution for each persistent calss, we obtain the TPM for a large m, say 100:
> P <- matrix(0,nrow=6,ncol=6)
> P[1,] <- c(1/3,2/3,0,0,0,0)
> P[2,] <- c(2/3,1/3,0,0,0,0)
> P[3,] <- c(0,0,1/4,3/4,0,0)
> P[4,] <- c(0,0,1/5,4/5,0,0)
> P[5,] <- c(1/4,0,1/4,0,1/4,1/4)
> P[6,] <- rep(1/6,6)
> rowSums(P)
[1] 1 1 1 1 1 1
> library(ACSWR)
Warning message:
package ‘ACSWR’ was built under R version 3.2.5
> msteptpm(P,100)
[,1] [,2] [,3] [,4] [,5] [,6]
[1,] 0.50 0.50 0.0000000 0.0000000 0.000000e+00 0.000000e+00
[2,] 0.50 0.50 0.0000000 0.0000000 0.000000e+00 0.000000e+00
[3,] 0.00 0.00 0.2105263 0.7894737 0.000000e+00 0.000000e+00
[4,] 0.00 0.00 0.2105263 0.7894737 0.000000e+00 0.000000e+00
[5,] 0.25 0.25 0.1052632 0.3947368 5.715143e-39 5.715143e-39
[6,] 0.25 0.25 0.1052632 0.3947368 3.810095e-39 3.810095e-39
The reciprocaal of bold numbers gives us the stationary distribution for each persistent class.
c) The expected number of steps needed to first return to state 1, conditioned on starting in state 1 is 1/0.5 = 2.
d) The expected number of steps needed to first reach state 2, conditioned on starting in state 1, is 1/0.5 = 2.
e). If we start in state 5, the probability of it eventually reaching state 2 is 0.25.
Related Questions
drjack9650@gmail.com
Navigate
Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.