StATS: What is an alpha level?

Alpha is the probability of making a Type I error (rejecting the null hypothesis when the null hypothesis is true). You want this value to be small, so you plan an experiment so that the sample size is large enough and the decision rule is selected so that alpha is 0.05 or sometimes 0.01.

There is insufficient discussion of when it is appropriate to change the alpha level from 0.05 to a slightly larger value or to a slightly smaller value. If the consequences of a Type I error are not all that important, then you might want to select a larger value for alpha to save some money or to optimize some of the other aspects of the research design (such as lowering beta, the probability of a Type II error).

If a Type I error is very serious, or if you have the luxury of having a very large sample size, then you might want to consider a smaller value for alpha.

Example: In a study of physicians self-reported levels of comfort and skill with treating children with emotional problems, the researchers deviated slightly from the norm of using an alpha level of 0.05.

To compare Comfort/Skill scores across condition types, we used repeated measures ANOVAs. In all analyses, we adopted a more stringent alpha level of 0.01 to indicate statistical significance (to protect against Type I error rate inflation).  -- Family physicians' involvement and self-reported comfort and skill in care of children with behavioral and emotional problems: a population-based survey. Miller AR, Johnston C, Klassen AF, Fine S, Papsdorf M. BMC Fam Pract 2005: 6(1); 12. [Medline] [Abstract] [Full text] [PDF]

This work is licensed under a Creative Commons Attribution 3.0 United States License. It was written by Steve Simon on 2005-08-12, edited by Steve Simon, and was last modified on 2010-04-01. This page needs minor revisions. Category: Definitions, Category: Hypothesis testing.