Know-How Exchange : Research/Metrics

In order to answer your question, you also need to know the margin of error that you'd be willing to tolerate. Here's an online calculator that you can use to help determine sample:

https://www.polarismr.com/education/tools_ss_prop.html

Note that the sample referred to in the calculator is the sample you end up with, not the number you actually poll (since some of those you poll presumably won't provide complete responses). So you also need to decide, based on what you know about your audience, an approximate number of those you would need to poll to result in the sample size you'd like to achieve for the significance testing.

Hope this helps - good luck!

jlevin,
Thank you for your answer.
I tried using one of those online calculators, but I'm unsure about trusting them.

If my entire population is 60,000 people (15,000 * 4 events), the sample size, according to the calculator should be 382.

But if I go with a population of 15,000 and try to determine how many to poll at each of the 4 events, the calculator says to sample 373.

So which is the correct answer? I mean, if I'm only speaking directly to 1/4 of the visitors at each event, wouldn't I run the risk of having too small a cell if I only poll 382 people at all 4 events?

I'm confused.

Thanks jlevin,
That's what I thought too. Good to have my thinking reinforced!

For calculating sample size, you really need to know three things: Population (in your case, it’s 15,000), confidence (you picked 95%), and margin of error. Margin of error is how close you require the sample results to actual population. If you choose 5% (common pick), you can say that the sample results reflect the actual population results within +- 5%. So, the calculator from the link Joy gave you would give you a sample of 375 to be 95% confident that your sample results are no more than 5% different from the true population results. If your sample results were that 37people did not like your event, than you could say that can say that with 95% confidence, between 5% to 15% of actual population would not like the event. If you want more precision, you need to increase the sample size. For instance, you’d need 997 people to be within 3%.

With this result, you could predict the results for the population. Your question, however, was how many people do I need to decide if the two groups are different. This is given from a z-test of proportions and you can find a calculator for this at:

https://www.dimensionresearch.com/resources/calculators/ztest.html

Because you have more information – two samples – you can get by with less people to be significant. If you play around with the calculator a bit, you can find that if you have 211 in each group, you can differentiate between the two samples to within +/-5% with a 95% confidence. That means as long as the results of the two samples are wider than 45%/55%, you can conclude that the samples are different. If you have 375 in each sample, you can differentiate down to 46%/54%. The nice thing about this tool is that you can calculate “on-the-fly” by putting the results in even when the sample sizes are different and when you have results to 95% that the sample results are different, you can stop. The interpretation here is that if you have results outside 45%/55%, you can conclude within 95% that the samples come from different populations. In your case, since we know they came from the same population, it may mean that direct contact versus viewing from afar makes a difference.

I hope this helps.

Wayde

Wayde brings up an excellent point here. I was assuming that you'd want both the ability to look at sample versus true population as well as the ability to test differences. If you just want to test the differences then you can get by with smaller sample, as his explanation and online tool points out.

Thanks all,

You're right jlevin, I wanted to be able to compare both ways - one segment to the other as well as to the general population.

So that leaves me with your last suggestion, right?