Suppose that a Markov chain has four states 1, 2, 3, 4, and stationary transition probabilities as specified by the following transition matrix
p = [1/4 1/4 0 1/2]
[0 1 0 0 ]
[1/2 0 1/2 0 ] [1/4 1/4 1/4 1/4]
a. If the chain is in state 3 at a given time n, what is the probability that it will be in state 2 at time n + 2?
b. If the chain is in state 1 at a given time n, what is the probability it will be in state 3 at time n + 3?