The simplest experiment we can model is one in which there are just two
possible outcomes. By convention, one of these is labelled or
and the other is labelled or .
Such experiments are referred to as Bernoulli trials. The corresponding
Bernoulli random variable assigns the value 1 to the outcome and
0 to . If denotes the probability of observing the
outcome labelled , then the p.m.f. of a Bernoulli random variable
is given by

The corresponding distribution function is,

Note that this distribution is characterized by the single parameter, .

The expected value and variance of a Bernoulli r.v. can be obtained easily:

and so,

Note that the s.d. of the Bernoulli is just

A direct extension of this experiment is an experiment in which a series
of independent Bernoulli trials are performed, each with the same
probability of success. Let denote the Bernoulli
random variable, and let denote the total number of successes among
the trials. Then,

Note that the gambler's problem described in the previous section is an example of this type of experiment in which represents the event that the first gambler wins a game and each game is a Bernoulli trial. Using the same arguments here as we did for the gambler's problem, we can see that the p.m.f. of is given by,

As noted earlier, the p.m.f of this random variable involves terms of the binomial series, and so this random variable is referred to as the binomial random variable and its distribution is referred to as the binomial distribution. This distribution is characterized by two parameters, the number of trials, and the success probability, .

The binomial distribution can be used to model a sampling experiment in which a sample of objects is to be randomly selected with replacement from a population that consists of two types of objects. Let denote the proportion of the population that are type 1 objects. The sampling can be viewed as a sequence of Bernoulli trials, with denoting the event that the first type is selected on a trial. Since the sampling is done with replacement, the second selection trial is identical to the first and the outcome of the second trial does not depend on the outcome of the first trial, since the object selected on the first trial is returned to the population. Hence, the number of type 1 objects selected in such an experiment would have a binomial distribution with trials and success probability .

If the sampling is performed without replacement, then the trials will no longer be independent and the probability of success for each trial will no longer be the same. However, if the sample size is small compared to the population size, then the binomial distribution is a reasonable approximation to the actual probabilities.

The expected value and variance can be derived directly from the Bernoulli
distribution.

Since can be expressed as a sum of independent Bernoulli r.v.'s, then the variance of this sum is the sum of the individual variances,

and so,

**Example**. Suppose that we are interested in the proportion of
defectives within a large population, and so randomly select a sample of size
from this population. If is small compared to the population size, then
we can use the binomial distribution as an approximation to the distribution of
the number of defectives that will be found in the sample. Let denote this
number and note that

represents the proportion of defectives in the sample. We would expect that the sample proportion should be fairly close to the population proportion . This can be expressed probabilistically by applying Chebychev's inequality. Let denote an arbitrarily small positive real number. Then,

This means that if is large, then the probability that the sample proportion is more that from the population proportion is very small. In fact, as , then this probability goes to 0.

A different extension of Bernoulli trials is to continue performing the
trials until the first success is observed. Let denote the
number of trials required to obtain the first success. Under the assumption
that the trials are independent with the same probability of success, we have,

This follows from the fact that the first success occurs on trial if and only if the first trials are failures and trial is a success. Since the p.m.f. of this random variable involves terms of the geometric series, this random variable is referred to as the geometric random variable and its distribution is referred to as the geometric distribution.

A related random variable that is sometimes more convenient to work with is
, defined to be the number of failures observed before the first
success occurs. It can be seen that is related to by
. Hence, its p.m.f. is given by,

The expected value and variance of the geometric distribution are:

2013-12-17