Limited Time Offer: Save 30% on PRO with code WOOHOO »

Real-World Education for Modern Marketers

Join Over 600,000 Marketing Professionals

Start here!
Text:  A A

Two Survey Questionnaire Flaws to Avoid: Scary Matrix Questions and Unbalanced Scales

by Carey V. Azzara  |  
July 27, 2010

Poorly designed questions and scaling problems can derail your research efforts faster than you can say "the cat in the hat"! 

To help you avoid a few of the more common and onerous problems, we will explore two separate but related questionnaire-design issues: matrix questions, especially the big, scary kind, and unbalanced scales, which provide data that is, at best, difficult to analyze and, at worst, useless.

Scary Matrix Questions

First, we examine what we will refer to as scary matrix questions (SMQs). A colleague of mine who likes that phrase said he imagined research professionals texting in acronyms: "OMG did U R the RI with all the SMQs? LOL." (Translation: Oh my god, did you read the research instrument with all the scary matrix questions? Laughing out loud.) Don't be the subject of such laughter.

The use of matrix questions that could scare a respondent into dropping out is counterproductive. Yet matrix questions are used frequently and written with less care than you would write an email.

Those considerable and overwhelming questions are used typically to collect a large number of data points in a relatively small amount of space.

One justification is to create a data set that will provide enormous differentiation using a large set of variables and values. The more likely outcome is tired respondents who straight-line their answers (i.e., select all 3s, 4s, or 5s) as a way to move through the task quickly.

What's the outcome? Rather than a highly differentiated set of responses, the research creates the equivalent of white bread and adds very little to the study's ability to gather insights.

Sign up for free to read the full article.Read the Full Article

Membership is required to access the full version of this how-to marketing article ... don't worry though, it's FREE!


We will never sell or rent your email address to anyone. We value your privacy. (We hate spam as much as you do.) See our privacy policy.

Sign in with one of your preferred accounts below:

Carey V. Azzara is the principal and founder of AtHeath, LLC (, a strategic research planning and consulting firm. He is the author of Questionnaire Design for Business Research (Tate Publishing, 2010). Reach him via

Rate this  

Overall rating

  • This has a 5 star rating
  • This has a 5 star rating
  • This has a 5 star rating
  • This has a 5 star rating
  • This has a 5 star rating
3 rating(s)

Add a Comment

MarketingProfs uses single
sign-on with Facebook, Twitter, Google and others to make subscribing and signing in easier for you. That's it, and nothing more! Rest assured that MarketingProfs: Your data is secure with MarketingProfs SocialSafe!