Next: Long-Run Behavior of Markov Up: Discrete Time Markov Chains Previous: State Probabilities

## Transition Probabilities

The one-step transition probability is the probability of transitioning from one state to another in a single step. The Markov chain is said to be time homogeneous if the transition probabilities from one state to another are independent of time index .

The transition probability matrix, , is the matrix consisting of the one-step transition probabilities, .

The -step transition probability is the probability of transitioning from state to state in steps.

The -step transition matrix whose elements are the -step transition probabilities is denoted as .

The -step transition probabilities can be found from the single-step transition probabilities as follows.

To transition from to in steps, the process can first transition from to in steps, and then transition from to in steps, where .

In matrix form, this becomes:

Setting yields:

From this equation we can see that:

Substituting this back into the previous equation yields:

Continuing these substitutions, eventually we have:

Therefore, the -step transition probability matrix can be found by multiplying the single-step probability matrix by itself times.

The state vector at time can also be found in terms of the transition probability matrix and the intial state vector . We first observe that:

In vector and matrix form, this becomes:

We also find that, through substitution:

or,

Continuing the substitution yields:

where is the vector containing the intial probabilities of being in each state at time 0.