The one-step transition probability is the probability of transitioning
from one state to another in a single step.
The Markov chain is said to be time homogeneous if the transition
probabilities from one state to another are independent of time index .
The transition probability matrix, , is the matrix consisting of the one-step transition probabilities, .
The -step transition probability is the probability of transitioning
from state to state in steps.
The -step transition probabilities can be found from the single-step transition probabilities as follows.
To transition from to in steps,
the process can first
transition from to in steps, and then
transition from to in steps, where .
The state vector at time can also be found in terms
of the transition probability matrix and the intial state
vector . We first observe that: