P.Mean: Where did that standard deviation come from? (created 2008-07-09).

Someone wanted some help with a power calculation. I gave the standard spiel that you need three things

  1. a research hypothesis,
  2. an estimate of the standard deviation of your outcome measure, and
  3. the minimum clinically important difference.

This was for a study looking at 10 exposed patients (recent spider bites) and 30 control patients. I got an article back in email very quickly, and while it was interesting to read, it wasn't quite what I needed.

Notice that these standard deviations are derived from fibroblast cultures. This make it an in vitro study--a study using Petri dishes. It is unrealistic to use this data to help plan an in vivo study--a study involving live human beings. The standard deviations from the in vitro study are probably too small, maybe one or two orders of magnitude too small. Those Petri dishes are coddled in a carefully controlled laboratory environment. Human beings live and breath in a more complex and heterogeneous environment.

It gets worse because it describes the standard deviations as technical replicates. That means that a single set of cells was extracted from a single Petri dish. This single sample was washed, heated, centrifuged, or whatever, and only at the last step was it split into several parts. The standard deviation obtained from technical replicates is almost always too small for planning a research study. 

This reminded me of the advice in an excellent paper on guidelines for sample size determination by Russ Lenth.

Historical data include data collected by the investigator in past experiments or work, and data obtained by browsing the literature. Historical or pilot data do not need to follow the same design as the planned study; but one must be careful that the right variance is being estimated. For example, the manufacturer of the sphygmomanometers to be used in the SBP experiment may have published test results that show that the standard deviation of the readings is 2.5 mm Hg. This figure is not appropriate for use in sample-size determination, because it probably reflects variations in readings made on the same subject under identical conditions.

The article, by the way, that I grabbed the table from is

Creative Commons License This work is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2010-04-01. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Sample size justification.