Question

Topic: Research/Metrics

Data Weighting

Posted by Anonymous on 6/10/2009 at 12:30 PM ET 125 Points
I have been working with client on weighting our data and a question has come up regarding the "industry practices."

With a typical survey, if I am trying to weight the data by age group, there is a group of respondents that typically refuse the question (in this case around 4%). With most weighting programs, those refusing the weighted question are not included in the weighted data. One way around that is to create a weight for "non-respondents/refusals" in addition to the age weightings. This way we don't throw out 4% of our responses.

What would be considered "standard industry practices" in this case -- to throw out the non-respondents to the age question, or to add a "non-respondent" weight along with the other age categories?
To continue reading this question and the solution, sign up ... it's free!
SUBSCRIBE
Or, SIGN IN

RESPONSES

  • Posted by koen.h.pauwels on 6/10/2009 at 2:58 PM Accepted
    creating the 'age unidentified' bucket would be best to keep the survey results representative. Also, do ask yourself which people refuse to answer this question and why (e.g. a certain gender above a certain age) to get some insight
  • Posted by BizConsult on 6/10/2009 at 11:34 PM Accepted
    Glenn:

    If there are non-respondents, you could include the uncooperative people within a non-respondent / refusal percentage, ignore them in the weighting, or run the math both ways so the client can select what best works for them.

    Much depends upon the research objectives, but I agree that deeper understanding of who the non-respondents are is useful, as no matter what you do above, if the “private” people tend to fall within a certain demographic cluster (on age, gender, income, etc.), no matter which output format you use, the results are skewed.

    You might also look at where, when and how the age question is asked: If it’s open ended, you’re likely to get less cooperation (or at least less honest participation!); If asked within broader age ranges (18-25, 26-33, etc.), that might elicit more response. Where within the survey is the question posed? If you haven’t yet convinced the survey takers that this is legit research, that might dampen response rates – i.e., if you haven’t gotten people to open up yet and are asking it as a screener question - that may eliminate more people.

    Also consider the follow-up from the survey takers to non-respondents (for example, “I’m sorry but we cannot use your information/responses in the research unless we can identify the proper age range, could you please identify which range you fall into?”, or similar.)

    Let us know where you net out!
    -Steve
  • Posted by mgoodman on 6/11/2009 at 4:15 PM Accepted
    Do you think that including (or excluding) the 4% of "refusers" would change the client's decisions? Most of the time not, so the question is largely academic.

    Go back to the stated research objectives and expected use of results. If the action standard had an implicit assumption that all respondents would provide age data, then you need to identify the group explicitly as "non-responders." Otherwise, who cares? It won't change the decision, which is what the client really needs to deal with.

    Remember that market research is a tool for supporting management decision-making, not an end in itself.
  • Posted by Christian on 6/12/2009 at 8:13 PM Member
    glenn,

    We had a discussion very similar to this today. It seems that one of the leading trends is to fill in the missing values with the mean of the values that you have.

    However this should be done carefully. For example, if you are segmenting based on age I would not include the missing values.

    Look at your sample size and cell size, do you need the 4% for a valid sample?

    I hope this helps!

    Cheers,
    Christian Vanek
    Co-Founder & CTO, SurveyGizmo
  • Posted on 5/20/2011 at 7:54 AM Member
    How to evaluate the performance of the 100 nos. of NGOs on the basis of following question asked to them.

    1st column : Name of Project/Activity
    2nd Column :Number of Seminars conducted
    3rd Column :Number of participants
    4th Column : Participant profile

BACK TO TOP

Post a Comment

Related Discussions

Community Info

Top 25 Experts
(Research/Metrics)