StATS: Goodness of fit test (May 18, 2004).

The chi-square test appears in a lot of different places. Some recent data on Astrology, published in the May newsletter of the Skeptic Society, offers an interesting opportunity to show one of these tests. In an article offering dark matter as a possible explanation of the effects of astrology, the London Times published a list of the 1,000 richest people and their star signs. It noted a significant difference between the number born under Gemini and the number born under Pisces. A careful look at the full data set, though, shows that numbers observed could easily be explained by sampling error.

There are actually 1,067 people listed here, which suggests either a typographical error or perhaps inflation has hit the top 1,000 list. The question is whether the distribution of star signs is uniform or not. A non-uniform distribution could be taken as evidence for Astrology, but it would probably need replication to be taken seriously.

If the distribution is uniform, we would expect to see about 89 people in each star sign. We compare the expected count to the observed count using the Chi-square goodness of fit statistic, X2:

where Oi are the observed counts (110, 104, 95, ..., 73) and Ei are the expected counts (88.92 for each star sign).

The table below shows the calculation of

You should compare this statistic to a Chi-squared distribution with 11 degrees of freedom. The 95th percentile of this distribution is 19.7. Since your test statistic is smaller than 19.7, you would accept the null hypothesis and conclude that the distribution is uniform. Actually, you should  state the conclusion more cautiously: there is insufficient evidence for a non-uniform distribution of star sign among the richest 1,000 people. You can also compute a p-value for this test statistic. In Microsoft Excel, the function CHIDIST(13.57,11) produces a p-value of 0.258.

Now if you just took the largest group, Gemini, and compared it to the smallest group, Pisces, you would indeed get a statistically significant differrence:

Compare this test statistic to a Chi-squared distribution with 1 degree of freedom, where the 95th percentile is 3.84. The p-value (CHIDIST(7.48,1)) is 0.0062.but there was no reason to believe a priori that Gemini would have a surplus of rich people compared to Pisces. Here's what the astrology.com web site says about Gemini:

Gemini is the third Sign of the Zodiac, and those born under this Sign will be quick to tell you all about it. That's because they love to talk! It's not just idle chatter with these folks, either. The driving force behind a Gemini's conversation is their mind. The Gemini-born are intellectually inclined, forever probing people and places in search of information. The more information a Gemini collects, the better. Sharing that information later on with those they love is also a lot of fun, for Geminis are supremely interested in developing their relationships. Dalliances with these folks are always enjoyable, since Geminis are bright, quick-witted and the proverbial life of the party. Even though their intellectual minds can rationalize forever and a day, Geminis also have a surplus of imagination waiting to be tapped. Can a Gemini be boring? Never!