P.Mean: How to design a new survey (created 2008-10-28).

This page is moving to a new website.

Someone wrote in with a question about how to design a survey. There are entire books devoted to the subject. I couldn't do the subject justice in a single email, but here's what I sent.

First, get some advice from experts in your area. Have them offer some qualitative impressions about your survey. Did you ask the questions properly? Are any questions likely to cause confusion or produce ambiguous responses? Are there any important areas that no question on this survey covers? This level of review can be described as establishing content validity for your survey. Some people use the term face validity, instead. As I understand it, face validity is established by average people like you and me while content validity is established by smarter people than you and me.

Second, lay out your survey carefully on a sheet of paper using a large font size. 12 point is very good, and 11 point is a bare minimum. For an elderly population, you might want to use 14 point or even 16 point fort size. How the survey looks will have a major impact. A clean layout will minimize errors and improve data quality. Make sure that the location where survey respondents are supposed to provide an answer is consistent. If the locations hop around between the left margin, right margin, and the middle, then survey respondents are more likely to accidentally skip an item.

Third, pilot test your survey. Have a small group of people fill out the survey as you watch them. Ask them to talk out loud about what they are thinking about when they fill out the survey. Notice any long pauses, any obvious misinterpretations of the wording of your questions, and any self-described confusion. After they are done, ask them what they thought about the survey.

This work is licensed under a Creative Commons Attribution 3.0 United States License. This page was written by Steve Simon and was last modified on 2010-04-01. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Survey design.