Selecting the right questions for your survey instruments can be tough. Equally difficult is identifying the right answer options for the questions you ask people. When selecting answer options, ideally you would provide enoughoptions to get some meaningful feedback and variation in responses, but not too many answer options as to overwhelm survey respondents.
Before launching any survey instrument it’s preferable to do what is called a survey pretest. A pretest is where you get a subsample of people who are like your intended survey audience, and you ask them for feedback on each of your survey questions and answer options. However, pretesting isn’t always possible.
I’ve been working with a nonprofit called Team Tassy that provides workforce services to families in Menelas, Haiti. Team Tassy wanted to learn more about the employability of families in its targeted communities by conducting a survey at a free medical clinic day the organization sponsored.
One of the questions on the survey asked what work related skills each of the respondents possessed. The problem was that we didn’t know whether we were providing the right answer options to the skills question.
Ideally we would have pretested the question to get feedback on what types of skills should be included in the answer options. However, pulling together a focus group abroad would have provided logistical challenges, making pretesting less of a viable option.
Since we were not able to pretest the answer options, instead Team Tassy took its best guess as to what the answer options should have been, and provided an option for respondents to fill in any other skills not included as part of the question’s answer options. We planned to use the free-form options to better learn what job skills options should have been included.
Team Tassy collected more than 250 surveys at the medical clinic it sponsored. Given the relatively large number of surveys, reading through each of the free-form answers wasn’t completely practical. Instead, we built a word cloud of the free-form skills options to get a visual idea of what types of skills were most mentioned.
The word cloud revealed that several individuals reported having merchant and dress making related skills, options that were not included among the original answer options. Going forward, Team Tassy will now include these options on future skills questions.
Word clouds are a pretty low-tech approach to data analysis. But they can be really effective, especially for getting quick feedback on what types of answer options you might included on your surveys.