next up previous
Next: Special Discrete Distributions Up: Class Notes Previous: Random Variables

Expectation of Discrete Random Variables

Suppose you are really bored one afternoon and decide to toss two coins a large number of times, say 10,000 times, recording the number of heads after each pair of coins is tossed. We can treat the tosses as binomial experiments with $n=2$ and $p=0.5$. Let $X_i$ denote the number of heads obtained on the $i^{th}$ toss. Then, we can expect after performing these experiments that around 2,500 of the $X_i$'s will be 0, around 5,000 of the $X_i$'s will be 1, and around 2,500 of the $X_i$'s will be 2. Now suppose we wish to find the average of the $X_i$'s, that is, the average number of heads per experiment. Based on the number of times we expect to observe each of the possible values, 0,1,2, the average we can expect to see would be,

\begin{eqnarray*}
\overline{X} &=& \frac{\sum X_i}{10000} \approx \frac{0*2500 +...
...(\frac{2500}{10000}\right)\\
&=& 0(.25) + 1(.50) + 2(.25) = 1.
\end{eqnarray*}

This quantity is referred to as the expected value of the random variable $X$, the number of heads when two coins are tossed. Note that the expected value represents the average we would expect to observe if an experiment is repeated a large number of times, just as the probability of an event is the proportion of times we would expect the event to occur if we repeated the experiment a large number of times. It is no coincidence that the expected value in this case coincides with

\begin{displaymath}
\sum_x xp(X=x) = \sum_x xp(x),
\end{displaymath}

where $p(x)$ is the p.m.f of $X$.

Definition. The expected value of a discrete random variable with p.m.f. $p$ is defined to be

\begin{displaymath}
E(X) = \sum_x xp(x).
\end{displaymath}

The expected value is also commonly referred to as the mean of the random variable. Properties of summation lead to the properties of expected values listed below. In this list, $X, Y$ represent random variables and $a, b, c$ represent constants.
  1. If $Y=a+bX$, then

    \begin{displaymath}
E(Y) = a + bE(X).
\end{displaymath}

  2. More generally, if

    \begin{displaymath}
Y = a_0 + \sum_{i=1}^n a_iX_i,
\end{displaymath}

    then

    \begin{displaymath}
E(Y) = a_0 + \sum_{i=1}^n a_iE(X_i).
\end{displaymath}

  3. Let $I_A(x)$ denote the indicator function for the set $A$. That is, $I_A(x)=1$ if $x\in A$ and $I_A(x)=0$ if $x\not\in A$. Then

    \begin{displaymath}
E[I_A(X)] = P(X\in A).
\end{displaymath}

  4. If $X$ and $Y$ are independent, then

    \begin{displaymath}
E[XY] = E[X]E[Y].
\end{displaymath}

The expected value of a r.v. describes the center of the possible values of the r.v. Note that it is a weighted average of those values in which the weights are given by the p.m.f. A related quantity, called the variance, describes the variability of those values about this center. Let $X$ be a discrete r.v. with expected value $E(X) = \mu$. Then the variance of $X$ is defined by:

\begin{eqnarray*}
Var(X) &=& E[(X - \mu)^2] \\
&=& \sum_x (x - \mu)^2p(x).
\end{eqnarray*}

The variance of a r.v. is commonly represented by $\sigma^2$. The square is because the unit of measure of the variance is the square of the unit of measure of the r.v.

Note that the properties of expectation imply:

\begin{eqnarray*}
Var(X) &=& E[X^2 - 2\mu X + \mu^2] \\
&=& E[X^2] - 2\mu E(X) + \mu^2 \\
&=& E[X^2] - \mu^2.
\end{eqnarray*}

This is usually the easiest way to obtain the variance.

The square root of the variance, called the standard deviation and denoted by $\sigma$, represents a measure of distance between a r.v. and its mean. In particular, since $E[cX] = cE[X]$, then

\begin{displaymath}
Var(cX) = E[c^2 X^2] - c^2\mu^2 = c^2Var(X),
\end{displaymath}

and so,

\begin{displaymath}
SD(cX) = \vert c\vert\sigma_X.
\end{displaymath}

Also, $\sigma_X = 0$ if and only if $X = \mu$ with probability 1. If the s.d. of a r.v. is small, then the r.v. tends to be close to its mean, but if the s.d. is large, then the r.v. tends to be farther from its mean. This is made more precise by Chebychev's inequality.

Chebychev's inequality: if $X$ is a random variable with mean $\mu$ and variance $\sigma^2$, then for any positive constant $\epsilon$,

\begin{displaymath}
P(\vert X-\mu\vert>\epsilon) \le \left(\frac{\sigma}{\epsilon}\right)^2.
\end{displaymath}

In particular,

\begin{eqnarray*}
P(\vert X - \mu\vert > 2\sigma) & \le & \frac{1}{4},\\
P(\vert X - \mu\vert > 3\sigma) & \le & \frac{1}{9}.\\
\end{eqnarray*}

Example. Suppose that a company receives a shipment of 50 new PC's, 4 of which are defective. Suppose that your cost center is given 5 of these PC's, assumed to be randomly selected from the shipment. Find the expected value and s.d. of the number of defective PC's received by the cost center.
solution. Let $N$ denote the number of defective PC's. We must first obtain the p.m.f. of $N$. Note that the sample space of $N$ is $0,1,2,3,4$.

\begin{eqnarray*}
p(0) &=& \frac{{46\choose 5}}{{50\choose 5}} = 0.64696 \\
p(1...
...00195 \\
p(4) &=& \frac{{46\choose 1}}{{50\choose 5}} = 0.00002
\end{eqnarray*}

Note the these probabilities sum to 1. The expected value and variance can be obtained most easily from the following table.

$x$ $p(x)$ $xp(x)$ $x^2$ $x^2p(x)$
0 0.64696 0 0 0
1 0.30808 0.30808 1 0.30808
2 0.04299 0.08598 4 0.17196
3 0.00195 0.00585 9 0.01755
4 0.00002 0.00008 16 0.00032
sum 1 0.4   0.49791


So, $E[N] = 0.4$, $E[N^2] = 0.49791$, $Var(N) = 0.49791 - 0.4^2 = 0.33791$, $SD(X) = \sqrt{0.33791} = 0.5813$.


next up previous
Next: Special Discrete Distributions Up: Class Notes Previous: Random Variables
Larry Ammann
2013-12-17