Question

Topic: Student Questions

Survey Whole Pool Not Randomised

Posted by Anonymous on 3/25/2008 at 6:57 PM ET 250 Points
Hi, I am doing grad research and I have a real survey going now, with a clearly defined pool size of exactly 1500. I have surveyed every person in the entire pool using online survey sent via email. So I have not made any randomisation decisions about representative samples. I have received 200 responses. (Figures rounded for this discussion).

A sampling calculator gives me 90% confidence at 5.4% error rate for these responses.

QUERY:

(1) Are there any issues about sending to the whole pool like this? I can't see why this would be a problem but a wiser head might know something to warn me about here.

(2) My responses
10 opted out.
70% who responded completed to the end.

(a) Do the semi-responders affect the overall picture and drag down the confidence level overall? (ie can I only really count those who got through to the end)? Do I delete the "opt outs" from the total number of rersponses?

(b) And when I discuss each question in my report, do I need to work out a separate CI for that question in terms of the whole pool size? Or do I mostly just talk about "of the 50 people who answered this question, 25% responded (etc), and flag any warnings about response level?

Very grateful for any advice.

Sally

To continue reading this question and the solution, sign up ... it's free!
SUBSCRIBE
Or, SIGN IN

RESPONSES

  • Posted on 3/27/2008 at 6:55 PM Accepted
    Hello Sally,
    Do clarify your “pool size” (the 1500 cases) is that your entire universe (population), or is this a sample of the universe (for example, if you had only 1500 email addresses for the universe of 2000)?

    To answer your first question. You are correct; there are no downsides to surveying the entire population. It would actually be the preferred thing to do for every study. Only because it is usually impossible, too costly or just unnecessary do we use sampling techniques.

    Answer 2a) Yes, you should ‘kill’ the opt-outs / incomplete cases; there are of course methods of dealing with / approximating the missing values (see: https://www.spss.com/missing_value/ ,do keep in mind that they are trying to sell you stuff :). As a rule, if the respondent did not see the ‘Thank You’ page at the end of the survey, their case has a status of incomplete, and therefore should be excluded from analysis.

    2b) The report should be a document that can be understood by people with little or no experience in statistics. It is your job as a researcher to analyze and interpret the data and later present your findings in such a way as to avoid ‘show your work’ paragraphs.
    If you run t-tests, you will know which, if any findings are significant (of course, any question with a base of 30 or less should be treated as an indicator only).

    I would also suggest that you use targeted weights for completed cases, so that your sample is more representative of the actual population make up. At least make sure that the male / female ratio matches between population and sample.

    Luke Zukowski
  • Posted on 3/31/2008 at 11:33 PM Author
    Thank you Luke for the helpful reply. Yes the "whole pool" is every single member of an entire industry in a particular state (they require registration to be in that industry). A few of these did not have emails addresses so I sent them hard copies.

    Thanks again
    Sally
BACK TO TOP

Post a Comment

Related Discussions

Community Info

Top 25 Experts
(Student Questions)