P.Mean: Comparing two proportions out of the same multinomial population (created 2008-08-05).

I am lucky enough to be researching wine. Specifically I am exploring which components in wine results in maximised preference. At the moment I am trying to compare proportions from the same population. N = 68. 8 people most preferred wine 1, 25 most preferred wine 2, 1 most preferred wine 1 and 2, for 34 of participants their most preferred wine was another wine. I want to see if the proportion of people that chose wine 1 was significantly different from the proportion that chose wine 2. I have been recommended to use McNemar's. But I just don't know how. I found your website which is as close as I have got but is slightly different. Just wondering if you had any thoughts? Cheers

First of all, let's arrange some more wine tasting and include me in the study. I'll have to warn you that a good experiment would require a very large sample size.

McNemar's sounds like the right test, but it is a total disaster. You have two proportions which share the same denominator, but which don't add up to 100% because there are a third proportion or maybe more. In mathematical terms, you are comparing two proportions from a single multinomial distribution.

Instead, suppose there were only two wines. Then testing whether a greater percentage preferred the first of the two wines would simply be a test of whether the first proportion was greater than 0.5. So why not just toss out wines #3, #4, etc. and reduce your problem to a simple binomial?

In your case, there are 33 people who express a preference for either wine 1 or wine 2 (let's ignore that troublemaker who liked both wines equally). Only 8 preferred wine #1. That's a proportion of 0.24. The 95% confidence interval is 0.10 to 0.39. Since that excludes 0.5, conclude that wine #1 is preferred by significantly fewer people than wine #2 among those who expressed a preference for either wine.

You can attack the problem from several other approaches, but I suspect they would all lead to the same answer.