Each second a laptop computer\'s wireless card reports the state of the communic

Question

Each second a laptop computer's wireless card reports the state of the communications channel to an access point. The channel may be in one of four states: 1 (poor), 2 (fair), 3 (good), and 4 (excellent). In the poor state, the next state is equally likely tobe poor or fair. For states 2, 3, and 4, the probability is .9 that the state will remain unchanged at the next second, and is .04 that the state will be poor at the next second. In states 2 and 3, the probability is .06 that the next state is one step up in quality. In state 4, the next state is either good with probability .04 or fair with probability .02. Provide the transition matrix P for the Markov chain defined by this process, and determine which states are recurrent. Is this chain ergodic?

Explanation / Answer

A simple two-state Markov chain A Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process usually characterized as memoryless: the next state depends only on the current state and not on the sequence of events that preceded it. This specific kind of "memorylessness" is called the Markov property. Markov chains have many applications as statistical models of real-world processes. Another example is the dietary habits of a creature who eats only grapes, cheese or lettuce, and whose dietary habits conform to the following rules: It eats exactly once a day. If it ate cheese today, tomorrow it will eat lettuce or grapes with equal probability. If it ate grapes today, tomorrow it will eat grapes with probability 1/10, cheese with probability 4/10 and lettuce with probability 5/10. If it ate lettuce today, it will not eat lettuce again tomorrow but will eat grapes with probability 4/10 or cheese with probability 6/10. This creature's eating habits can be modeled with a Markov chain since its choice tomorrow depends solely on what it ate today, not what it ate yesterday or even farther in the past. One statistical property that could be calculated is the expected percentage, over a long period, of the days on which the creature will eat grapes. A series of independent events (for example, a series of coin flips) satisfies the formal definition of a Markov chain. However, the theory is usually applied only when the probability distribution of the next step depends non-trivially on the current state. Many other examples of Markov chains exist. [edit]Formal definition A Markov chain is a sequence of random variables X1, X2, X3, ... with the Markov property, namely that, given the present state, the future and past states are independent. Formally, The possible values of Xi form a countable set S called the state space of the chain. Markov chains are often described by a directed graph, where the edges are labeled by the probabilities of going from one state to the other states. [edit]Variations Continuous-time Markov processes have a continuous index. Time-homogeneous Markov chains (or stationary Markov chains) are processes where for all n. The probability of the transition is independent of n. A Markov chain of order m (or a Markov chain with memory m), where m is finite, is a process satisfying In other words, the future state depends on the past m states. It is possible to construct a chain (Yn) from (Xn) which has the 'classical' Markov property by taking as state space the ordered m-tuples of X values, ie. Yn = (Xn, Xn-1, ..., Xn-m+1). An additive Markov chain of order m is determined by an additive conditional probability, The value f(xn,xn-r,r) is the additive contribution of the variable xn-r to the conditional probability.

Related Questions

Navigate

• Browse (All)
• Browse E
• Subjects

---

---

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.