Differences between revisions 12 and 13

Statistical Tests for Experiments with Single Samples

Sometimes, the research question is as simple as investigating whether the value of a certain property of a group differs from a hypothetical value (specified by the null hypothesis). Depending on what information is available, one can choose between the following types of tests:

In this section, we are going to review these tests by working through a few toy problems.

Z-test

Single-sample t-test

The single-sample t-test is often more commonly used as we as experimenters rarely know the true standard deviation of the target population. The only available information has to come from the sample data collected in the experiment. For example, Smokin' Joe wants to know whether marijuana can increase appetite and conducted an experiment:

Example Problem

Smokin’ Joe hypothesizes that marijuana can be used to mitigate some of the negative side effects of common AIDS drugs, such as loss of appetite. He wants to test the hypothesis that marijuana increases the appetites of AIDS patients who are taking common AIDS drugs. He measures the difference in calories eaten by patients the day after taking a THC pill and the day before taking the pill. Here are the measurements he obtains for 10 subjects – [101, 75, -82, 32, -50, 203, 165, 145, 303, 23]

In other words, Smokin' Joe wants to test whether the difference in calories eaten by AIDS patients (i.e. the property of a group) is different from 0 (i.e. a hypothetical value). If it is significantly different from 0, we can reject the null hypothesis which states that marijuana has no effect on appetite.

Note here the only information available is the difference scores of the 10 subjects. We do not know the true standard deviation of the population of difference scores. Therefore, a z-test is not applicable in this case.

Approximate Population SD with Sample SD

One possible solution to the problem, which allows us to use the single-sample t-test, is to approximate the population standard deviation $\sigma$ with sample standard deviation $s$ . Recall that the equation to calculate the z statistic $z_{obt}$ is:

$z_{obt} = \frac{\overline{X}_{obt} - \mu}{\sigma / \sqrt{N}}$

In the single sample $t$ test, we simply replace $\sigma$ with the standard deviation of the sample $s$ to get $t_{obt}$ :

$t_{obt} = \frac{\overline{X}_{obt} - \mu}{s / \sqrt{N}}$

Once we have calculated $t_{obt}$ , we need to determine the significance of the statistic. If you are working with a stats program, the significance $p$ value, including the $t_{obt}$ , is calculated automatically for you. If you are working with pen and paper, you will need to look up in the t distribution table (usually in the appendix of any statistics textbook) for the corresponding $t_{crit}$ value. In looking for the correct critical value to compare with, we also need to determine the degrees of freedom (DF) of the current test. For any single-sample t test, the DF is $N-1$ , where N is the sample size. It is then easy to locate the critical value in the table by choosing the right test (one-tailed or two-tailed?), the $\alpha$ level, and the DF.

Solve the Example Problem By Hand

Now we proceed to solve the example problem. Let's do a two-tailed test (see GeneralGuidance for topics on how to choose between a two-tailed test and a one-tailed test) with $\alpha$ value set to 0.05. To calculate the $t_{obt}$ , we calculate the following items first:

The sample average difference score $\overline{X}_{obt} = 91.5$
The population average difference score $\mu = 0$ (by definition; since we want to know whether there is a change)
The sample standard deviation

Note: always carry at least three decimal places if working by hand. Rounding excessively in intermediate steps can lead to very very very inaccurate results.

Now we plug these numbers in the above equation:

$t_{obt} = \frac{\overline{X}_{obt} - \mu}{s / \sqrt{N}} = \frac{91.5-0}{117.449 / 3.162} = 2.463$

Looking up in the critical values table for the t distribution, we find that $t_{crit} = 2.262$ given that DF is 9 and $\alpha$ is 0.05 for a two-tailed test. Since our t statistic is greater than the critical value, we reject the null hypothesis. We conclude that Smokin' Joe's experimental results do not support the hypothesis that marijuana does not increase appetite.

It is extremely important to keep in mind that we cannot conclude that the experiments results support the alternative hypothesis, which states marijuana does increase appetite. The rejection of the null hypothesis, in frequentist statistics (which t-test belongs to), does not mean that the alternative hypothesis is TRUE.

Sign test (using binomial distribution)

The sign test is based on the binomial distribution

-  ⇤ ← Revision 12 as of 2011-09-03 15:41:26 → 
  Size: 5370
  Editor: cpe-69-207-82-159
  Comment:
+   ← Revision 13 as of 2011-09-04 01:25:47 → ⇥
  Size: 5471
  Editor: cpe-69-207-82-159
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
- * [[#sign-test | sign test]]
-Line 10:
+Line 8:
-In this section, we are going to review these two tests by working through a few toy problems.
+ * [[#sign-test |sign test: when the experimental results are binary outcomes]]
-Line 12:
+Line 10:
-<<Anchor(sign-test)>>
== Sign test (using binomial distribution) ==
+In this section, we are going to review these tests by working through a few toy problems.
-Line 18:
+Line 14:
-Line 74:
+Line 71:
+<<Anchor(sign-test)>>
== Sign test (using binomial distribution) ==

The sign test is based on the binomial distribution