The state probability, denoted as , is the probability
that the process is in state at time .

The state probability vector is denoted as , and consists of
all of the state probabilities for a given time .

Note that the sum over the elements in is equal to 1.