Additional Properties of Probability

The complement of an event is defined to be the set of all outcomes contained in the sample space that are not contained in the event. It is denoted by $A^c$. Note that the complement of the sample space is defined to be the empty set, $\emptyset$, the set with no elements. Also, $A\bigcup\emptyset = A$ and $A\bigcap\emptyset = \emptyset$ for any event A. Therefore, if we set $A_1 = A$, $A_i = \emptyset,\ i\ge 2$, then $\{A_i\}$ is a countable collection of mutually exclusive events. Hence, from axiom 3 we have,

$$P(A) = P(\bigcup_{i=1}^\infty A_i) = P(A) + \sum_{i=2}^\infty P(\emptyset).$$

Since $P(\emptyset)\ge 0$ from Axiom 1, then this equation implies that $P(\emptyset)=0$.

Note: mathematical equations are sentences with the same syntax as English and can be read as such. The set operations, intersection, union, and complement are often read as the English equivalents, and, or, and not, respectively. Also, the word or used in this context is assumed to mean the inclusive or.

Now let $A_i,\ 1\le i\le n$ be a finite collection of mutually exclusive events and set $A_i=\emptyset$ for $i>n$. Then from Axiom 3, we have

$$\begin{eqnarray*} P(\bigcup_{i=1}^n A_i) &=& P(\bigcup_{i=1}^\infty A_i) \\ &=... ...+ \sum_{i=n+1}^\infty P(\emptyset) \\ &=& \sum_{i=1}^n P(A_i). \end{eqnarray*}$$

That is, the probability of a finite union of mutually exclusive events equals the sum of the probabilities.

Suppose we are interested in an experiment in which the sample space consists of a finite collection of $n$ outcomes, $O_i,\ 1\le i\le n$, and that each outcome is equally likely with probability $p$. Then the previous result implies that

$$1 = \sum_{i=1}^n P(O_i) = np.$$

Therefore, we must have $p = 1/n$. Furthermore, since an event for such an experiment may be written as the union of the individual outcomes contained in the event, then

$$P(A) = \frac{\char93 \{A\}}{n},$$

where $\char93 \{A\}$ represents the number of elements in the set A. For experiments with equally likely outcomes, the probability of an event is just the number of outcomes in the event divided by the total number of outcomes.

Next note that A and $A^c$ are mutually exclusive and $A\bigcup A^c = S$. Therefore, from the previous result we have,

$$1 = P(A\bigcup A^c) = P(A) + P(A^c).$$

So, the probability of the complement of an event is one minus the probability of the event. This result is useful for situations in which an event of interest is very complicated and its probability is difficult to obtain directly, but the complement of the event is simple with an easily obtainable probability.

The axioms of probability tell us how to find the probability of the union of mutually exclusive events, but not how to find the probability of the union of arbitrary, not necessarily mutually exclusive, events. We can use the results derived thus far to solve this problem. Suppose we are interested in two events, A and B. We need to write the union of these two events as the union of two mutually exclusive events. This can be done by noting that $A\bigcup B = A\bigcup \{B\bigcap A^c\}$. Since A and $B\bigcap A^c$ are mutually exclusive, then

$$P(A\bigcup B) = P(A) + P(B\bigcap A^c).$$

Next note that $B=\{A\bigcap B\}\bigcup \{B\bigcap A^c\}$, which is a disjoint union. Therefore,

$$P(B) = P(A\bigcap B) + P(B\bigcap A^c)$$

and so,

$$P(B\bigcap A^c) = P(B) - P(A\bigcap B).$$

Combining this with the previous result gives,

$$P(A\bigcup B) = P(A) + P(B) - P(A\bigcap B),$$

the probability of A union B equals the sum of the probabilities minus the probability of the intersection.

In a similar way, we can show that probability is a monotone function. Suppose that $A\subset B$. Then we may express B as a disjoint union, $B=A\bigcup(B\bigcap A^c)$ and apply the additivity property of probability,

$$\begin{eqnarray*} P(B) &=& P(A\bigcup(B\bigcap A^c)) \\ &=& P(A) + P(B\bigcap A^c) \\ &\ge& P(A), \end{eqnarray*}$$

since $P(B\bigcap A^c)\ge 0$. Hence, if $A\subset B$, then $P(A)\le P(B)$.

Another extension that can be derived directly from the axioms is an extremely useful result called the Law of Total Probability. A partition of the sample space is defined to be a collection, finite or countably infinite, of mutually exclusive events in the probability space whose union is the sample space. Suppose that $\{B_i\}$ is partition and A is an arbitrary event. Then $A = \bigcup \{A\bigcap B_i\}$, and the events, $A\bigcap B_i$ are mutually exclusive. The Law of Total Probability is just the application of Axiom 3 to this expression,

$$P(A) = \sum P(A\bigcap B_i).$$

This property allows us to breakdown a complicated event A into more manageable pieces, $A\bigcap B_i$.

Example. Suppose a standard card deck (13 denominations in 4 suits) is well-shuffled and then the top card is discarded. What is the probability that the $2^{nd}$ card (the new top card) is an ace? Let $A$ enote the event that the $2^{nd}$ card is an ace. The partitioning events we will use are the events

$$B_1 = \{1^{st}\ {\rm card\ is\ Ace}\},\ \ B_2 = \{1^{st}\ {\rm card\ is\ not\ Ace}\}$$

Then,

$$\begin{eqnarray*} P(A) &=& P(A\bigcap B_1) + P(A\bigcap B_2)\\ &=& P(1^{st}\ {... ...\ {\rm card\ is\ not\ Ace} \bigcap 2^{nd}\ {\rm card\ is\ Ace}). \end{eqnarray*}$$

The first term has numerator which is the number of ways the first card is an ace and the second card is an ace, and has denominator which is the total number of different outcomes for the first two cards. We can use permutations to count the number of outcomes for both numerator and denominator. The numerator is $4\cdot 3$ and the denominator is $52\cdot 51$. Hence,

$$P(1^{st}\ {\rm card\ is\ Ace} \bigcap 2^{nd}\ {\rm card\ is\ Ace}) = \frac{(4)(3)}{(52)(51)}.$$

Similarly, the second term is

$$P(1^{st}\ {\rm card\ is\ not\ Ace} \bigcap 2^{nd}\ {\rm card\ is\ Ace}) = \frac{(48)(4)}{(52)(51)}.$$

These give

$$\begin{eqnarray*} P(A) &=& \frac{(4)(3)}{(52)(51)} + \frac{(48)(4)}{(52)(51)}\\ ... ...=& \frac{(4)(51)}{(52)(51)}\\ &=& \frac{4}{52} = \frac{1}{13}. \end{eqnarray*}$$

Note that this probability is the same as the probability that the first card is an ace.