Know-How Exchange : Research/Metrics

Thanks Research/Metrics.

My target demography is 18 - 34 year olds and in the Vancouver,canada region. I just did a search on myspace and found out that there are atleast 30000 people in the myspace database belonging to my target demography.
So ideally I should be alright if I have a web based survey with myspace users.
I dont know how to calculate sample size and I'm just going by my gut for 1000 people.
I was thinking may be I could have sweepstake prizes, with one being something worth $500 and another 10 being $50 worth amounting to a total of $1000 ($500 +$50 X 10).
Also the early bird, first 1000 entries get one free song download along with the an entry to sweepstakes.

To control being scammed I could let only 1 entry from a particular IP address.
The whole thing would cost me $2000.

Any thoughts on this?

If the myspace crowd is your demo then what do you have to lose? But how will you get them to take a survey? You still have to buy advertising to get your message across. Not knowing your target demo I might try contacting some of the other social network type sites and see if you can get the ads cheaper or maybe they would do a survey for you.

You could use surveymonkey.com or as RM mentioned Zoomerang to create your survey.

WNelson:I think putting a project up here is always an option.

Harry: Surveymoney.com looks more attractive at $19.95/ month as opposed to $599 yearly subscription with zoomerang.

I think I just might be able to pull this off.

Keep the suggestions coming :)

Wayde: That really helps!
So, for a 95% confidence level and 5% margin of error I would only need 385 samples and...
correct me if I am wrong here, with 1038 samples I can get 99% confidence level and 4% margin of error then?

That would bring me to the next question.
whats the standard confidence level used for market surveys or does it depend on industry and demographics etc?

Yes you used the calculator correctly. The answer to "standard confidence level and margin of error" (you didn't ask, but I assume you are interested in that too) is that it depends....

How much does it cost you if you are wrong? How much does it cost for more samples? You weigh the two against each other and go with it.

If I were setting up a marketing survey, the 95%/5% would be a reasonable place to be. And even if it's 90%/10% - depending on what I'm looking for, I'd take that too - especially if I am doing multiple serial surveys. That way, I'd preserve my budget for additional surveys that home in on certain areas that I derive from the initial ones.

So, I'd shoot for 385 samples for the first try - see the results, and then you would have two more shots at it if you have other questions.

Where you SHOULD spend a LOT of resources is on the questions. Designing surveys is science and art. You want to think through all of the possible interpretations of words and phrases and then you want to "test" the survey before you launch to make sure you get what you need. Designing a survey is not something you can post here and get a reasonable answer or a reference book that you can read and make you an expert overnight. Again, it all depends on how important it is to get good data as to how much you spend on this surveying. If you're talking about a "hobby" with little or no economic impact, go for it. Otherwise, you may want to seek some help. (My last cautionary note! :) )


Wayde

There is some real good stuff in here. Thanks so much guys!!

ViaBuzz: Can you please give me some more information on the search robot companies. I kinda like the idea but am not able to fully comprehend how to go about it. If you can give me a name of one company that does it, I can research it even more. Right now the google search for "search robot companies " isnt every helpful.

My other question is, for this demography of 18 - 34, how does one go about allocating age quotas for the survey. Is it 50 % each for 18 - 24 and 25 - 34 or should I go more finer than that with 30 something percent each for 18-24, 25-39, 30-34.
Does it matter?

I am going to get the questionnaire reviewed by a marketing professional and with regards to getting scammed by sweepstakes folks (Thanks Denny) , I think I will have a landing page which will only let referrals(users) from a particular domain (Myspace) access the survey. This way, only the people who have clicked on a survey link they received on their myspace mail will be able to access the survey.

Frank: Thanks. I hope I am able to tap into the myspace resource and get something worthwhile.

Unless you suspect your survey or survey method will bias your sample or that your sample population (myspace) is not representative of the age distribution of the total population, you don't need to worry about gathering a defined proportion in your sample. In fact, the very effort to "fix" the sample distribution, will make it NOT representative of the population and therefore biased. And if myspace is not representative of the population of interest for you, then you may want to reconsider using it.

Wayde

I feel there is a very good chance that people who use myspace would actually be among those who would use my product at the end of the day. Would that be considered biased?
Myspace does have the right demography that I am looking to target with the product and the survey. There are atleast 30,000 people between the ages of 18 and 34 from my target region.

My Concern is that the reward (Song download + chance to win $200) may not be as attractive to a 31 year old as it would be to a 19 year old and since its on a first come first serve basis, all the enthusiastic 19 year olds would have filled up majority of the 385 spots
and I may not get enough feedback from the 29 - 34 year olds for this.
So I was thinking by having X% quota alloted for each group I can have make sure that I get input from all different age groups.


P.S: Thanks Wade. You are being very helpful with this. :)

Remember I told you that you don't blow your total budget on a one-shot sample so you have more to spend if you find questions in the results you get and want to sample again? This is why. First thing you want to do is get some census data of the population mix by age so you know what the distribution looks like of 28-24 versus 25-31. This way you can judge if your offer is yielding a representative sample of the population. While this isn't brain surgery, the test for this, is not for the novice statistician. Perhaps your pro can help you with this too. Then what you want to do is complete your survey (designed by the pro) and then do a focus group, say, for a sample of 10 (local) people in ages representative to the population. In addition to the survey, ask them some questions about their reaction to the survey questions (clarity, ease, etc), and their preferences for the download offer versus other things you might use to entice participants. Offer them an enticement for participating in the focus group - maybe a $20 gift certificate or something like that. Then, based on the answers they give, redesign the survey and offer accordingly. After that, you launch a pilot survey of, say, 100 with your offer and then take a look at the results in terms of the age distribution versus your understanding of the general population from your census research. There are statistical tests for this. If it's representative, then go for another 285 as is. If not, redesign the survey and/or offer (maybe after another focus group session specifically aimed at the group that is disproportionally low). Then, pilot again, test, etc. I'd budget for a three focus group sessions, two 100 person pilots, followed by another 100 and then the remaining 285 (in other words, while I believe that I might hit it right after the first focus group, I'd rather have budget that I didn't use than not enough - this would be the worst case where I have to test 100 twice and find it's not representative of the population twice, get it right the third time, and complete the sample work). This plan would be representative to what you would do if the survey results are critical to your project results. If the value of the information is high (because the cost of product/service development and launch are very high), then spend the time and effort to get this right. If you are just trying to gain some insight to confirm what you think you already know based on extensive feedback that you already have, then blow it off, design a survey on the cheap, execute it, analyze it, and close your eyes to the details that you may not have quite the mix of ages representative to the population. Judge this by how many 25-31 year olds there really are versus 18-24 and how attractive you believe your product and service would be to 25-31 yr olds in general. If you are missing their input but they probably would buy your product and service anyway, then who cares, right? It's an economic decision, not a question of statistical purity.

Wayde

Thats super Wayde. Thanks. I think I am going to make that my action plan. I am going to meet up with my marketing guy in the next couple of days and work this out.

Have a good weekend! :)

Sorry i was away. Thanks Carrie and Thank you everyone for al your help.

Super appreciate!