**[StATS]:** **Should I use the heteroscedastic
t-test? (July 26, 2007)**

*Dear Professor Mean, I am using Excel to do a t-test to see if there
is a difference between blood glucose levels in babies who received 2
different treatments. I want to know if the new treatment increased
blood glucose levels. When I use the t- test function in Excel, it
asks me homoscedastic or heteroscedastic? Can you help?*

I’m going to provide a good news/bad news answer to your question.
The bad news is that there is no consensus on this in the research
community. You will get **different opinions about which test to use
depending on which statistician you ask**. The good news is that no
matter which choice you make, I can find an expert who will tell you
that you made the right choice.

My current recommendation is to use the homoscedastic test unless you have a strong a priori reason to believe that the data come from two populations with markedly different standard deviations. For what it’s worth, I have made a variety of different recommendations in the past as my thinking on this issue has evolved.

First, let’s take a look at the Excel help file.

The choice that you are asking about, of course, is between type=2 and
type=3. **This function produces a p-value** for testing the
hypothesis that the two data sets come from populations with the same
population mean. Unfortunately, **Excel doesn’t produce a confidence
interval for the difference between the two means**, which is usually
of more interest to researchers. So you would be better off using
SPSS.

But in SPSS you would still have the same dilemma. SPSS doesn’t make you chose when you select the dialog box for the t-test. It presents both versions of the test.

The **homoscedastic test** (note other terms/spelling, such as equal
variances, homoscedasticity, homoskedastic, homoskedasticity) **places
an extra assumption on your back**. It requires that the two
populations that I just referred to have to have the same population
standard deviation. This is in addition to the assumption that all the
data values are independent and the assumption that the data follow a
normal distribution (the classic bell shaped curve).

The **heteroscedastic test** (note other terms/spelling, such as
unequal variances, heteroscedasticity, heteroskedastic,
heteroskedasticity) **saves you having to make that assumption**. The
individual data values still have to be independent and they still
have to come from normal distributions, but with the heteroscedastic
test, the two populations could have different population standard
deviations and the test would still be valid. Sometimes statisticians
call this the Behrens-Fisher problem. Sometimes they call the test the
Satterthwaite approximation and sometimes the Welch-Satterthwaite
approximation. That’s just because we like to honor some of the
people who’ve made contributions in this area.

There are **two competing issues** here, **validity and power**. If
the test is invalid that means that the sampling distribution that we
have proposed for the test statistic is not the correct one and the
alpha level of the test may be different that what you intended it to
be. But when both tests are valid, the homoscedastic test will have
more precision and power because it uses a larger value for the
degrees of freedom.

So no strategy is risk free. The **conservative position is to always
use the heteroscedastic test**. This approach makes the implicit
assumption that protecting the alpha level of the test is always more
important than protecting the power of the test. It is also a very
simple approach, and because of this simplicity, I have advocated the
conservative position from time to time.

The **liberal position is to use the homoscedastic test unless
circumstances demand otherwise**. There are three different ways of
assessing when the circumstances are severe.

Some people **use the homoscedastic test unless a particular test of
homoscedasticity, such as Levene’s test is rejected**. This is the
approach that SPSS implicitly encourages, because it presents
Levene’s test right next to the choices for homoscedastic and
heteroscedastic tests. I dislike this approach because Levene’s test
is sensitive to the sample size and may reject too often for large
samples sizes and too infrequently for small sample size. There is no
evidence that the validity of the homoscedastic test is affected by
sample size.

Another way to assess the circumstances is to use a cut-off that is
independent of the sample size. In the past, I have recommended that
you examine the ratio of the two sample standard deviations and **only
switch to the heteroscedastic test if the two sample standard
deviations differ by a factor of 3 or more**. I don’t have any
empirical evidence for this approach, but I suspect it would work
better than the above approach.

This work is licensed under a Creative Commons Attribution 3.0 United States License. It was written by Steve Simon.

This page was written by Steve Simon while working at Children’s Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children’s Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at

for pages similar to this one at with general help resources. You can also browse Children’s Mercy Hospital website. Need more information? I have a page reproducing it here as a service, as it is no longer available on the Hospital. Although I do not hold the copyright for this material, I am This page was written by Steve Simon while working at Children’s Mercy