Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Survey says: Be careful what you ask for

With the ability to quickly conjure up an online interactive survey thanks to software as a service (SaaS) technology, any developer or business can almost instantly start polling potentially hundreds or millions of respondents with dozens of questions. Do you really need all that data?

Designing surveys whose questions do not introduce bias or ambiguity on the part of the survey sponsor is not an easy task. It’s a skill, a profession. You’ve got to figure out what questions to ask and how to word them properly. You need to provide for all possible answers (including “prefer not to answer” and “don’t know/don’t care”). You can’t allow answer choices to overlap (0-10 and 10-20 instead of 11-20). If you’re tasked with building a survey, you might want to brush up on the seven sins of survey question writing and how to avoid them. Of course, there are hundreds of other online resources.

Questioning the questions might not be an app developer’s job, but your logical mind is likely better-suited for finding the kinds of mistakes that could render a survey’s results useless.

What got me thinking about applying a developer’s keen eye to the logic of survey questions? It’s the surprisingly invasive nature of a survey I just took on the subject of interchangeable lens DSLR (digital single-lens reflex) cameras.

At 40 minutes, the survey was way too long and complex. But, what really bothered me was the invasive nature of some introductory questions: race, marital status, household income, number of children, technology devices in my home (including Segway — really?), and my favorite, “general statements about different attitudes you might have toward life in general.”

Are these really germane to a survey on cameras and lenses? Perhaps. Even if they are valid from a statistical analysis perspective, they sure seem nosy. Let me put it another way. If this was a survey about which cloud application development tools you like best, would you be willing to answer those very same questions?

Have you been asked to build surveys for your company? Did you find logic errors or other problems with any of the questions? If so, did you speak up? Share your thoughts; we’d like to hear from you.