Ex. 1 - Consider the two-state DTMC with the transition matrix P = [ 1 - a a b 1 - b ] a, b ∈ [0,1]. Discuss the reducibility, periodicity and limiting/stationary/occupancy distributions to classify the DTMC for different a, b.

Ex. 1 - Consider the two-state DTMC with the transition matrix P = [ 1

$2.28. Consider the weather model of Conceptual Problem 2.13. Compute the longrun fraction of days that are sunny. 2.13. Consider the following weather model. The weather normally behaves as in Example 2.3. However, when the cloudy spell lasts for two or more days, it continues to be cloudy for another day with probability .8 or turns rainy with probability .2. Develop a four-state DTMC model to describe this behavior. Example 2.3. (Weather Model). The weather in the city of Heavenly is classified as sunny, cloudy, or rainy. Suppose that tomorrow’s weather depends only on today’s weather as follows: if it is sunny today, it is cloudy tomorrow with probability .3 and 2.2 Examples of Markov Models 9 rainy with probability . 2 ; if it is cloudy today, it is sunny tomorrow with probability .5 and rainy with probability .3 ; and finally, if it is rainy today, it is sunny tomorrow with probability .4 and cloudy with probability . 5 . Model the weather process as a DTMC. Let Xn be the weather conditions in Heavenly on day n, defined as follows: Xn = { 1 if it is sunny on day n, 2 if it is cloudy on day n, 3 if it is rainy on day n. Then we are told that {Xn, n ≥ 0} is a DTMC with state space {1, 2, 3}. We next compute its transition matrix. We are given that p1,2 = .3 and p1,3 = .2. We are not explicitly given p1,1. We use p1,1 + p1,2 + p1,3 = 1 to obtain p1,1 = .5. Similarly, we can obtain p2,2 and p3,3. This yields the transition probability matrix P = [.50 .30 .20 .50 .20 .30 .40 .50 .10]$ $2.28. Consider the weather model of Conceptual Problem 2.13. Compute the longrun fraction of days that are sunny. 2.13. Consider the following weather model. The weather normally behaves as in Example 2.3. However, when the cloudy spell lasts for two or more days, it continues to be cloudy for another day with probability .8 or turns rainy with probability .2. Develop a four-state DTMC model to describe this behavior. Example 2.3. (Weather Model). The weather in the city of Heavenly is classified as sunny, cloudy, or rainy. Suppose that tomorrow’s weather depends only on today’s weather as follows: if it is sunny today, it is cloudy tomorrow with probability .3 and 2.2 Examples of Markov Models 9 rainy with probability . 2 ; if it is cloudy today, it is sunny tomorrow with probability .5 and rainy with probability .3 ; and finally, if it is rainy today, it is sunny tomorrow with probability .4 and cloudy with probability . 5 . Model the weather process as a DTMC. Let Xn be the weather conditions in Heavenly on day n, defined as follows: Xn = { 1 if it is sunny on day n, 2 if it is cloudy on day n, 3 if it is rainy on day n. Then we are told that {Xn, n ≥ 0} is a DTMC with state space {1, 2, 3}. We next compute its transition matrix. We are given that p1,2 = .3 and p1,3 = .2. We are not explicitly given p1,1. We use p1,1 + p1,2 + p1,3 = 1 to obtain p1,1 = .5. Similarly, we can obtain p2,2 and p3,3. This yields the transition probability matrix P = [.50 .30 .20 .50 .20 .30 .40 .50 .10]$ $2.28. Consider the weather model of Conceptual Problem 2.13. Compute the longrun fraction of days that are sunny. 2.13. Consider the following weather model. The weather normally behaves as in Example 2.3. However, when the cloudy spell lasts for two or more days, it continues to be cloudy for another day with probability .8 or turns rainy with probability .2. Develop a four-state DTMC model to describe this behavior. Example 2.3. (Weather Model). The weather in the city of Heavenly is classified as sunny, cloudy, or rainy. Suppose that tomorrow’s weather depends only on today’s weather as follows: if it is sunny today, it is cloudy tomorrow with probability .3 and 2.2 Examples of Markov Models 9 rainy with probability . 2 ; if it is cloudy today, it is sunny tomorrow with probability .5 and rainy with probability .3 ; and finally, if it is rainy today, it is sunny tomorrow with probability .4 and cloudy with probability . 5 . Model the weather process as a DTMC. Let Xn be the weather conditions in Heavenly on day n, defined as follows: Xn = { 1 if it is sunny on day n, 2 if it is cloudy on day n, 3 if it is rainy on day n. Then we are told that {Xn, n ≥ 0} is a DTMC with state space {1, 2, 3}. We next compute its transition matrix. We are given that p1,2 = .3 and p1,3 = .2. We are not explicitly given p1,1. We use p1,1 + p1,2 + p1,3 = 1 to obtain p1,1 = .5. Similarly, we can obtain p2,2 and p3,3. This yields the transition probability matrix P = [.50 .30 .20 .50 .20 .30 .40 .50 .10]$

No results found