The Do’s and Don’ts of
Questionnaire / Survey Design

Too often people believe they can write a great survey instrument without knowing the
ins and outs of survey instrument design. You can create your own survey instrument
if you follow a few simple rules.

GB rule 1 – Know the objective of the study

It’s vital to have a very clear understanding of what information you want to gain from conducting the survey. Consider searching online for similar survey instruments conducted for your topic or objective. Types of surveys include customer satisfaction studies, employee surveys, brand awareness investigations, and voice of the customer models, to name a few.
Never ask only open-ended questions.

GB rule 2 – Hypothesize what some outcomes might look like so you can set pre-coded responses to topics your respondent has said is important to them.

Using closed-end questions, in part, helps reduce the length of time it takes for the respondent to answer the survey. Consider doing a secondary research search about your topic prior to question design.

GB rule 3 – So many rating scales to choose from.

Which rating scale is best?

The answer will vary from one market research expert to another.

To be safe, our recommendation is that a stratum (number of responses) of 3 or less responses is too few, while 7 or more responses per question, in our opinion, is too many.

Do not use an unbalanced scale. Example…. A five-point scale that shows only positive responses and one okay response. In other words don’t stack the deck. In the end, you will be doing yourself and your company a disservice.

Extremely Satisfied
Very Satisfied
Slightly Satisfied
Satisfied
Fair
Poor

Use this scale instead:
Very Satisfied
Satisfied
Neither
Dissatisfied
Very Dissatisfied

We recommend using a five-point scale that uses a Likert scale … an equal number of positive and negative responses– neutral response in the middle.

GB rule 4 – Keeping the survey to less than seven minutes regardless of
the type of methodology.

GB rule 5 – Knowing what type of audience you want to measure (demographics, attitudes, users/non-users), and how detailed responses need to be (extensive open-ended questions versus closed-ended questions) designed. How the question is worded can bias the study.

Millennials for example prefer answering surveys on their mobile device.
In some cases, mix mode methodologies, while more difficult might be the only way to reach a good representative sample.

GB rule 6 – Don’t skimp on the number of people (Sample/ Responses)

Our general rule for traditional telephone surveys is that a survey seeking 400 completed responses, requires a sample ten times larger in order to generate 400 responses ( so, in order to record 400 completed responses a survey would need a 4000 respondents sample). For internet type surveys, the ratio can vary from a minimum of 20 to 1 and up.

Sampling methodology (always try to randomize your sample) and sample error are topics too complex for this post but there are many articles on how to accurately pull random samples. There are also free sample/response calculators now available online that can help you determine how many responses you need for your results to be scientific.

GB rule7 – To Incentivize the respondent/or not.

Depending upon how scientific your survey has to be, we recommend using a non-monetary incentive, such as a raffle,
especially for-profit studies. So many companies conduct their own survey these days. Response rates are at an all-time low. Any incentive you can use to increase participation is usually worth the additional cost.

GB rule 8 – Send out Reminders and Thank You notices…. Always.

Our recommendation is no more than two reminder notices.

GB rule 9 – Test your survey before going live.

Make sure you conduct a few surveys before you go live to make sure the instrument and process is working, especially
skips patterns in the survey.

GB rule 10 – Reporting /Analysis

Attached is an example of the type of “tabulations you might find when the study is finished.

Always make sure that each cell of the cross tab has enough responses to the report. Small cell sample sizes can lead to erroneous insights and recommendations.

Survey responses, unless weighted to a specific population, are not projectable to the US population.

We recommend never reporting any results with an N (number of responses by cell) of less than 20 for a survey of 400 total responses, and less than 50 for a survey of 1000 + responses.

The results of your survey should be able to tell a story that leads to opportunities for improvement in product design, operations process (customer satisfaction) brand awareness, or to measure product satisfaction just to name a few topics.

Art versus Science

While anyone can design a survey, there is a right way and a wrong way. And, designing an effective survey, choosing a methodology, and analyzing the results are a combination of art and science.

Conducting a survey is not inexpensive and why most small business owners choose to use market research vendors or tools such as Survey Monkey and Survey Gizmo which are free for small sample sizes. The capabilities of these programs are
not as robust as other survey tools (such as Qualtrics or Survey Tracker) – GB conducted a competitive analysis on internet tools and found Qualtrics to be the optimum internet program for us.

If you find designing your own survey is complex, or you don’t have the time or resources to conduct the survey yourself, there are plenty of certified market research firms you can help you conduct the right survey program for your
business objective.

We hope this information is helpful.

Need help? We would like you to consider GB & Partners for all your survey needs.

V/r.

George Brezny
Founder
GB & Partners
727-570-9000
gb-insights.com