Writing Successful Surveys 101
Author: Inge De Bleecker
Date: June 24th 2011
Using Surveys
Surveys can be a very efficient way of gathering information from people who are not present in person, or who reside in different locations or time zones.
Surveys can be used to gather different types of user information. At uTest, surveys can collect usage information, usability feedback, comments on functionality, performance, etc…
Understanding your Goal
Before you start writing survey questions, it is important to understand what the goal of the survey is; you may have some specific problems or questions, and gathering data will help you analyze these questions and come up with solutions.
For instance, a company may be getting a lot of customer support calls about a specific aspect of a product, without getting any real indication of any obvious reasons for these problems. Allowing uTest testers to use the product and fill out a survey may help identify issues. Or a company could be getting lots of reports about, say, slow download speeds that cannot easily be reproduced by the company. To find out what the scope of this issue is, a survey to a number of uTest testers is a good way of gathering information.
A survey can also help a company understand the value that some functionality has to users, and what future functionality is perceived as useful to be added to the product. Again, a survey can help gather that information.
Once you understand your goal, you can determine the following:
- number of survey takers
- length of survey
- types of survey questions
Determining the Number of Survey Invitees
There is no good general rule that tells you how many people to invite to your survey. This number greatly depends on the type of survey you are conducting and the types of questions and answers you are looking to collect and analyze.
Length of Survey
One of your main goals is to ensure that users complete the survey. Incentives can help completion rates, and if your survey takers have some vested interest in completing the survey, your completion rates will obviously be higher. The longer the survey, the higher the chance that survey takers will abandon the effort, unless the incentives increase accordingly. Most online usability studies take between 15 and 45 minutes to complete, but user’s attention will start to fade after about 20 or 30 minutes1. In more structured environments with committed users (e.g. testing at uTest) the completion rates are quite high for surveys that take quite a bit longer to fill out. A good rule of thumb is to keep your survey as short as you can, without compromising the data gathering process. If you must, run multiple surveys to gather all the information you are looking for.
1 Albert, Bill and Tom Tullis and Donna Tedesco. 2010. Beyond the Usability Lab. Conducting Large-scale Online User Experience Studies. Morgan Kaufmann Publishers, Burlington MA.
