MarketingProfs' Members Register for B2B Forum 2010 for just $695! (good until 11/30) »

Ask your question ... sign up today! It's FREE!
Research/Metrics     
 
This question has been closed, and points have been awarded.
Ranking V Rating Question Design
Posted By: Pint Glass on 11/3/2009 3:56 AM (CST) 125 Points
Hi All,

i am currently designing a client survey and I am having a few reservations about one particular question.

The question asks respondents to "rank the following 7 service attributes in order of importance. Please consider 1 to be the most important, 2 the next important etc. Remember no two items can have the same ranking."

The reason for this particular question is that a lot has changed in our particular industry in the last 12 months and I would like to find out what attributes are important to a client when choosing a service provider.

MY questions to you all are as follows:

-Will respondents fully understand this question
-Is there a better way of posing the question
-Am i better off using a rating scale for each attribute

Looking forward to all responses,

Pint Glass

Posted by: koen.h.pauwels Accepted Answer
11/3/2009 6:38 AM (CST)
Dear Pint,

I believe most respondents do understand a ranking question, and see no problems in your industry in this regard. However, you can experiment with a few different ways for getting the information you need:

a) evaluate EACH of the service attributes (eg out of 10, or 100, or indicate the importance on a 1-7 scale)
b) DIVIDE 100 points over the following 7 service attributes
c) evaluate following service PROFILES that combine different levels of the attributes (conjoint analysis)

a) is easy for respondents to fill out as it requires little thought (which increases response rates but may yield lower quality of answers)
b) forces respondents to carefully consider how much each attribute really contributes to their purchase decision. Therefore, it takes more time, but may yield information that better represents their evalution process. Typical example is price, which often gets the highest score in a), but often ends up a lot less important than other attributes in b) and c)
c) is more representative of how people evaluate competing offers, but may be way too cumbersome with 7 attributes. Can you reduce them to 4-5 key ones with 2-3 levels each?

Practically, I would work out the above (especially a and b) and present them to a few respondents you know very well. They can give you feedback about which is better, after which you can conduct your survey
 
Posted by: jlevin Member Response
11/3/2009 9:40 AM (CST)
I would also go with ranking, otherwise you can come up with a situation where everything is important. One way to force respondents into a choice is to allocate points to each attribute as koen.h.pauwels mentions above.

Depending upon how long the survey already is, you may also want to have respondents rate both your client and existing competitors on how they perform on each attribute.
 
Posted by: stevea Accepted Answer
11/5/2009 6:58 AM (CST)
Dear Pint

I too think that a ranked question would work as it forces the recipient to evaluate their priorities. Even if they do this off the top of their head there is research which states that gut instincts are actually more accurate in a survey than careful reasoning.

Another reason for going with a ranked question is that setting separate questions each with a separate score would make the questionnaire 6 questions longer than it need be. Participants shortly followed by you will have to unravel potentially conflicting answers and come up with a valid statistical treatment of the results.

Juliet (She’s in the top 100 here) and I used SurveyGizmo whose directors are also members of this site to do a survey with several ranked questions which was sent out to school teachers. They confounded our estimates of a 10% response rate by contributing about a 37% response. We attributed this excellent result to the subject under consideration and the fact that the questionnaire was easy to complete – ranked questions included.

The ranked questions were hardest to evaluate, but to give the varying opinions the correct weighting we multiplied the number of responses with their ranking in order to skew the answers more dramatically from strongly dislike though to strongly like. This is simply akin to cranking up the gain in a system and does not change the interpretation of the result – it simply makes it easier to see!

Good luck

Steve Alker
Xspirt
 
Posted by: Pint Glass Author Response
11/5/2009 8:12 AM (CST)
Hi Folks,

Thanks for all of your insightful and knowledgable comments.

I am going to close this now and I look forward to speaking with you in the future.

Pint Glass
 


Get more answers ... ReTweet this!

Would you like to post a response?
Welcome to Know-How Exchange!
This is a collaborative community. We welcome everyone's participation.
All you need to do is login. Enter your account info in the box above (top right).
Not a member? Not a problem. Register here (it's FREE and EASY).




Know-How Exchange powered by MarketingProfs


Sponsored Links

Buy a Link Now

Just Released ... Download Today!
Just Released ... Download Today!
Look inside
Included with Premium Membership
User Name:
Password:
Remember Me
Forgot your password?

Top 25 KHE Experts
(Research/Metrics)
ASVP/ChrisB (14554)
Jay Hamilton-Roth (11934)
jlevin (11599)
mgoodman (10158)
koen.h.pauwels (9157)
wnelson (7755)
DWomack (6993)
Frank Hurtte (6158)
stevea (6094)
skoobie99 (4979)
Peter (henna gaijin) (4267)
mbarber (4049)
Dawson (3995)
blanalytics (3883)
thinkmor (3634)
telemoxie (3021)
SteveByrneBranding (2755)
michael (2740)
darcy.moen (2727)
Michele (2419)
SRyan ;] (2396)
adammjw (2243)
holm (2169)
AndrewS (2081)
Wiglaf (2016)
Recently Posted Marketing Jobs
Director of Marketing and Communications
Demand Generation Manager
Marketing/Advertising Faculty
Director of Marketing
Market Analyst
Sr. Field Marketing Manager - Business Intell.
Associate Vice President of Marketing and Corporat
Marketing Manager
[more jobs]

Join over 355,000 members ... SIGN UP!

My email address is and I'd like my password to be .

Already a member? Sign In!

My email address is , and my password is .


HACKER SAFE certified sites prevent over 99.9% of hacker crime.