Statistical Tests for Experiments with Two Samples

A common experimental design is taking measurements from both a test group and a control group. For example, we may want to test the effectiveness of a new drug. A scientifically reasonable way of conducting an experiment is to give the drug to the test group while giving the placebo to the control group. If there is a significant difference between the test group and the control group, we can conclude the drug is at least statistically improving patients' well-being.

An important concept in experiments with two groups is "repeated measure design" or "paired design". A paired design refers to studies where the two or more sets of measurements are taken from a single group of subjects under different conditions. Suppose we want to study the effect of sleep on memory. Having a test group of subjects who sleep normally and another group of subjects who are deprived of sleep does not reveal much about the target effect since there may be inherent difference in memory performance between these two groups. Rather, a repeated measure design is recommended here so that we can measure memory performance when the group of subjects sleep normally and compare it to data collected after the same group of subjects are deprived of sleep.

In the following, we illustrate how to conduct statistical tests in the unpaired design (i.e. test group vs control group) and in the paired design.

Unpaired/Independent Group t-test

The unpaired t-test is applicable when the experimental group and the control group are different. Consider the following example:

Paired/Correlated/Repeated Group t-test

Example Problem

You are interested in determining whether an experimental birth control pill has the side effect of changing blood pressure. You randomly sample ten women from the city in which you live. You give five of them a placebo for a month and then measure their blood pressure. Then you switch them to the birth control pill for a month and again measure their blood pressure. The other five women receive the same treatment except that they are given the birth control pill first for a month, followed by the placebo for a month. The blood pressure readings are shown here.

Subject No.	Birth Control pill	Placebo pill
1	108	102
2	76	68
3	69	66
4	78	71
5	74	76
6	85	80
7	79	82
8	78	79
9	80	78
10	81	85

The first step, again, is to identify the null hypothesis: Birth control will not affect the blood pressure of women who are taking it ( $\mu_{D} = 0$ ). This also justifies a two-tailed test since we are not interested in whether the effect is directional (i.e. whether the birth control pill will increase or decrease blood pressure).

Solve the Problem by Hand

Now we test for a significant change in the blood pressure of women on birth control pills using an $\alpha = .05$ significance level.

First, calculate the mean blood pressure change of the sample:

$x_{obt} = 2.1$

Next, the standard deviation of the sample difference scores (this is done by taking the differences first and then calculate the standard deviation):

$s = 4.383$

Now, the standard error of the mean:

$s_{\overline{x}} = \frac{s_{x}}{\sqrt{n}} = \frac{4.383}{\sqrt{10}} = 1.386$

Note that in the unpaired t-test, we calculated the standard error of the difference in means, while here we calculated the standard error of the mean difference. This is the key difference between the independent group design and the correlated group design.

And, finally, our $t_{obt}$ :

latex error! exitcode was 1 (signal 0), transscript follows:

This is pdfTeX, Version 3.14159265-2.6-1.40.19 (TeX Live 2019/dev/Debian) (preloaded format=latex)
entering extended mode
(./latex_956fe02ca3bbbe600812f9bd36b9c010469c4175_p.tex
LaTeX2e <2018-12-01>
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2018/09/03 v1.4i Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size12.clo))
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty)
No file latex_956fe02ca3bbbe600812f9bd36b9c010469c4175_p.aux.
)
Runaway argument?
{s_{\overline {x \end {document} 
! File ended while scanning use of \frac.
<inserted text> 
                \par 
<*> ...fe02ca3bbbe600812f9bd36b9c010469c4175_p.tex
                                                  
! Emergency stop.
<*> ...fe02ca3bbbe600812f9bd36b9c010469c4175_p.tex
                                                  
No pages of output.
Transcript written on latex_956fe02ca3bbbe600812f9bd36b9c010469c4175_p.log.

= \frac{2.1 -0 }{1.386} = 1.515\] }}}

What is the degrees of freedom for a paired t-test? Since we only have 10 subjects in our group, the DF is the same as in the one-sample t-test: N-1. Now, we can look up critical value of t for a two-tail test using the alpha value 0.05 and the appropriate degrees of freedom (df = 10-1 = 9). The critical value is 2.262, which is larger than our obtained t statistic. Thus, we retain the null hypothesis and conclude that there is no sufficient evidence for the claim that the birth control pill affects the blood pressure of women who take it.