6. How to Design a Questionnaire 

(Textbook Page 189)

This textbook unit provides the principles, techniques and knowledge of the potential pitfalls in constructing a useable questionnaire – one of the most important things a marketing researcher is expected to do.  

Why is questionnaire design Important? (P. 190) 

• A well-written questionnaire will: collect information needed to achieve each survey objective, minimize the effort required by the respondent, in a way suitable for the methodology, with questions that will be interpreted the same way by all respondents and produce results that can be interpreted clearly. 
• The questionnaire writer must create standardized question wording and programming logic so nothing is left to individual interpretation.  There is no space for the researcher to interpret individual responses separately. 

Impact of Current Trends (P. 191)

• The prevalence of mobile technology and social media mean that attention spans are shorter than ever.  We need to minimize respondent burden and engage them in varied and exciting tasks.  At the same time we have the ability to capture consumer feedback any time and ‘in-the-moment’.  Social media has created a culture of self-exposure in which consumers open up about their deepest motivations. 
• Fraudulent respondents are a growing concern that must be taken into account. 

What are the steps? (P. 192)

1.    Clarify the Scope: List the topics needed to meet the research objectives, limiting it to ‘need to know’ topics to stay within budget and questionnaire length limits. Use this to create the questionnaire outline and to set the client’s expectations.   Map each topic (and eventually question) back to a specific objective and hypothesis. 

2.    Decide how the questions will be sequenced: achieve the required tasks, making respondents comfortable revealing themselves, being careful not to influence how they answer subsequent questions.  Generally they follow this sequence: 

a.    Screening – to ensure we interview the right people, asking only what is absolutely needed to qualify respondents and fill quotas to get the right mix.  
b.    Behavioural – what they do, where they go, what they own, how they make decisions, etc.  They are easy to answer and put respondents in the right frame of mind.  
c.    Attitudinal – what they think or feel.  Harder to answer because it may not be something they’ve thought about before.
d.    Lifestyle – how they live (may not be included in all questionnaires)
e.    Classification – who they are, if not needed up front for screening and quotas.  Collecting private or sensitive information once respondents are comfortable and more trusting.  If no decision will be made on a demographic question, don’t include it. 

Some important sequencing concepts: 

• Funnelling means moving from general to more specific questions, creating a common context to ensure each question is interpreted the same way.
• Building rapport and confidence by starting with questions that are simple, factual and easy to answer. Avoid complex grid or repetitive statement batteries at the beginning if possible.
• Avoiding fatigue by ensuring the questionnaire is short, not repetitive or confusing, and doesn’t require too much effort. 

3.    Wording the Questions (P. 203): This is critical to the quality of the data we get back.  It must :

o    encourage respondents to read/listen 
o    transmit the same meaning to all
o    make them feel comfortable answering truthfully. 

• This textbook unit provides tips on how to: 

o    Use simple words.  Start with how you’d ask a friend in conversation.
o    Be specific. Provide a list of acceptable responses. Use everyday consumer language. Avoid acronyms. 
o    Avoid difficult-to-answer factual questions: information the respondent never had, may not recall, may have to work too hard to find or may be to embarrassed to share.  
o    Avoid double barrelled question (have two parts but ask for one answer). 
o    Avoid ambiguous questions (can be interpreted in different ways by different respondents)
o    Avoid leading and loaded questions (leads respondents to a specific answer or implies a certain answer is socially or emotionally more desirable)
o    Avoid black-and-white when answers may be grey. Ensure all valid responses are possible.
o    Be careful when wording sensitive questions by developing a proper context.
o    Anticipate and counter respondent biases, such as social correctness, politeness, mind-reading and perceived importance.
o    Consider numeric responses carefully 
o    Be spare, edit to precision to ensure respondents read every word and avoid scrolling on small screens 

• This textbook unit describes the different question formats and when to use them: 

o    Closed-ended, including dichotomous and multichotomous questions, prompted and unprompted, pre-coded and non-coded
o    Open-ended, letting respondents determine the scope of the answer creating verbatim responses that are coded later
o    Image and video capture and analytics

4.    Questionnaire Layout and Design (p229): The look and feel of the questionnaire impacts respondents’ willingness to participate.  Appealing design increases attention, reduces boredom and improves data quality.  Some tips: 

o    Follow a respondent’s logic – walking through thought processes in order
o    Put questions respondents would consider important early in the questionnaire and complicated or potentially objectionable questions as late as possible.
o    Group questions that are similar in content together to create a natural flow
o    Signal when you change direction to transition respondents’ state of mind.

This textbook unit provides specific formatting tips that would:

o    Minimize respondent effort with straightforward design and brevity
o    Ensure instructions and flow are self-explanatory
o    Improve readability and comprehension for all respondents 
o    Use visual elements effectively

5.    Improving Data Quality in self-administered Surveys (p240): A small proportion of respondents are intent on earning any incentives as quickly as possible and will lie to qualify and then speed through, compromising data quality. 

• Techniques to catch poor quality respondents include inserting red herrings, identifying straight-lining, setting traps, flagging nonsense text responses, impossibly short completion times (speeding), overly easy recruiting and inconsistency between responses.
• Data quality is improved with higher the response rates.  Beyond good design, appropriate incentives, on-going engagement with a panel community and reminders can help.

6.    The Benefits of Online Questionnaires (P.244):  Online questionnaires:

o    give us better control over skip patterns and rotations, 
o    can employ different response methods and design to make them more engaging,
o    can personalize the content of specific questions, 
o    do a better job of hiding questions until they are to be seen, 
o    make it easier to change one’s responses, 
o    allow piecemeal completion and 
o    upload respondent’s answers immediately. 

7.    The Challenges of Multi-Country Questionnaire Design (p.248):   Global consistency requires an awareness of the challenges, including: 

o    Unique words, phrases or expressions in cultures that speak the same language
o    Using appropriate references to social roles (e.g. Head of Household)
o    Being aware of concepts (e.g. social class) that simply don’t exist in every culture
o    Understanding what may be offensive or deeply personal in each culture
o    Translating a questionnaire to get both the meaning and the tone right

8.    Reviewing, Refining and Pre-testing (P.250):  A final check of the questionnaire length is a wise disaster check. 

• Use the objectives map to identify and edit or omit superfluous (‘nice to know’) or redundant questions. 
• Once finalized it should be pre-tested, informally (e.g. testing with colleagues, swivel chair, protocol methods or link testing) or formally (e.g. testing with a small sample / soft-launch). 
• Finally, get formal sign-offs from all major stakeholders to ensure they are on board, and signal that no more alterations can be made.