next up previous
Next: Chi-square test for independence Up: Statistical Decisions Previous: Hypothesis tests to compare

Hypothesis tests to compare means of two populations - paired samples

Although separate samples selected from the two populations seems like a natural experimental design for this problem, this design may not produce the best results. In the salary comparison example discussed in the previous section, it would be reasonable to assume that the salaries of individuals in both populations would be related to experience. It may have been the case that the differences we saw in salaries was due to the fact that salary is strongly related to experience and there was a significant difference in experience between the two groups. We cannot eliminate the possible effect of experience and the confusion in interpretation that it causes by our independent sampling design.

By incorporating this additional variable into our experimental design we can reduce variability and potential confusion, thereby increasing the sensitivity of our test. There are two ways this can be accomplished. One way is to include a measure of experience as a second variable that is recorded for each person selected in the two samples. This design involves a type of analysis called Analysis of Covariance which is beyond the scope of this course. The second way to incorporate this additional variable into the analysis is to design a matched pairs sampling process. In this sampling design we randomly select one individual from the first population, determine the experience level of the person selected, then identify from the second population all individuals who have the same experience and randomly select one of those to be matched with the first individual selected. This selecting and matching process is continued to we obtain $n$ matched pairs of individuals, matched according to experience. This paired sampling design removes the effect of experience on salaries, and so any differences that remain between the two groups cannot be due to differences in experience.

Other comparison problems involve naturally occurring pairs. A common experimental design to test for the effect of some treatment is to give a pre-treatment test to a group, apply the treatment, and then give a post-treatment test. The test scores are naturally paired - one pre-treatment and one post-treatment score for each individual. In another example, we may wish to obtain a sample of married couples and then compare the scores of the husbands and wives on a questionnaire each takes. The key difference here between paired sampling and independent sampling is that in this case we are randomly selecting a sample of married couples rather than randomly selecting a sample of males who are husbands and separately selecting a sample of females who happen to be wives.

Let $(X_1,Y_1),\cdots,(X_n,Y_n)$ denote the pairs of measurements that are obtained from a paired-sample experiment, and suppose that we wish to test the hypotheses

\begin{eqnarray*}
H_0:&\ \mu_1 = \mu_2\\
H_1:&\ \mu_1 \ne \mu_2,
\end{eqnarray*}

where the first population has mean $\mu_1$ and the second population has mean $\mu_2$. In paired-sample experiments the pair differences, $d_i=X_i-Y_i$, $1\le i\le n$, contain the relevant information about the hypotheses. Therefore, we can express the hypotheses in terms of the pair differences by defining $\mu_d = \mu_1-\mu_2$. Then we have

\begin{eqnarray*}
H_0:&\ \mu_d = 0\\
H_1:&\ \mu_d \ne 0.
\end{eqnarray*}

This has the effect of transforming the problem to a one-sample problem, and so we can use the one-sample t-test applied to the pair differences to test these hypotheses. This test is based on the mean and standard deviation of the pair differences, $d_1,\cdots,d_n$. While it is possible to obtain the mean of the pair differences by computing the difference between the sample means, there is no such shortcut to obtain the standard deviation of the pair differences. It must be computed from the pair differences directly. The resulting test is called a paired-sample t-test, and the sample size for the test is the number of pairs.

Example Suppose that we implement a matched pairs sampling design for the comparison of salaries, matching on experience, and find that in a sample of 18 matched males and females, their salary information is
$\bar{X}_1=75200$, $s_1=5800$, $\bar{X}_2=72700$, $s_2=7000$, $s_d=4000$.
Note that the individual group standard deviations, $s_1,s_2$, are not part of the paired-sample test, and the sample mean of the pair differences is

\begin{displaymath}
\bar{X}_d = \bar{X}_1 - \bar{X}_2 = 2500.
\end{displaymath}

Therefore, the test statistic is

\begin{displaymath}
T_0 = \frac{2500}{4000/\sqrt{18}} = 2.652
\end{displaymath}

We use the t-distribution with 17 degrees of freedom since the data for this test consists of the 18 pair differences. The test statistic is between the t-values 2.567 and 2.898 which correspond to $t_{.010}$ and $t_{.005}$, respecively. Since this is a two-sided test, the p-value is between 0.02 and 0.01. Therefore, our decision is to reject the null hypothesis at the 5% level of significance. We might make a Type I error with this decision, but the chance of that happening is between 1% and 2%. A 95% confidence interval for the difference between the mean salaries is

\begin{displaymath}
\bar{X}_d \pm ts_d/\sqrt{n}\ \Longleftrightarrow\ 2500 \pm (...
...eftrightarrow\ 2500 \pm 1989\ \Longleftrightarrow\ [611,4489].
\end{displaymath}

We are 95% confident that the difference between the mean salary for males and the mean salary for females is in this interval. We can also say that this difference is not due to possible differences in experience between the two groups since the effect of that variable has been removed by our matched-pair sampling design.


next up previous
Next: Chi-square test for independence Up: Statistical Decisions Previous: Hypothesis tests to compare
Larry Ammann
2014-10-21