Know-How Exchange : Research/Metrics

The answers to your questions depend on the objective and expected use of the research results. These key up-front decisions will trigger the right methodology, and that will determine the cost and timing, sample size, etc.

Sample size. Sample size isn't usually expressed as a percentage of the market, but rather as an absolute number. That's because the accuracy/statistical variability of your findings are going to be based on such a small percentage of the total universe that the size of the universe doesn't much matter.

The sample size will also vary based on how many "slices" you'll want of the data. For example, if you want to look at men and women separately, then you'll need a large enough base so that each cell is going to give you a representative and projectable sample. Same with income splits, age splits, education level splits, etc., etc.

Longevity. Longevity depends on the product category/industry. In fast-paced markets (computer hardware/software, for example), the life of the data is short -- usually less than a year. In some industrial chemicals (for example), it can be good for years. And there's a range inbetween. Again, it depends.

Duration. And survey duration depends on the methodology. If you're doing it online, you should be able to get the actual field work completed in days (or perhaps a week). If you're doing a product usage test with extended use required, mailed forms, and sequential questions, it could be weeks. It depends. "Months" sounds too long to me for most categories/industries.

You might want to consider attending a seminar that MarketingProfs is sponsoring on June 22. The title/subject is Do-It-Yourself Market Research. There will be time at the end for questions and answers, and we'll probably be able to anticipate many of your questions. We'll also suggest some affordable professional help you can get in the planning stage.

A final note: Good market research is not something that first-time practitioners can usually pull off, and if the issues you're addressing are really important to you, you might want to think about the value an outside expert can bring to the party -- at least in the all-important planning stages. They can certainly help you with the kinds of questions you're asking.

Hope this helps. There are no simple answers to your questions. Everything depends.

First of all, you do not state what your market make-up is i.e. High tech versus industrial, consumer versus b2b, premium versus commodity. Some market studies can have value 4 or 5 years down the road while others are extremely short term. If you are talking about more stable industrial markets, I would spend the money upfront to do an exhaustive research effort and then budget an update every 18 months to be sure things are not changing.

1) In commonly accepted market research what is the standard sampling size expressed as a percentage of the overall test market?
I would contrat with a statistician adept at survey techniques to help you with this. Many projections of behavior for hundreds of millions are based on as few as 1000 data points but they are points chosen knowing they will give a reflection of the target market. You will need at least 35 good surveys in each market you plan to address from my experience. Less than that you are likely to be influenced by one or two. More than that depends on what you want to spend. Declining return for those dollars though - statistically.

2) What is the longevity of the data? At what point does this data become antiquated and questionable?
See my preamble. Find someone in your market you can ask this question of - they might give you some insight.

3) What is an acceptable survey duration? In summary planning I was looking into a 2-3 month duration, is this too long, too short?
Why does length of survey matter? I would say that you need to reach the critical number of responses. Shorter is always better. Go fast, spend less, and then use money to go back and check first learnings to confirm.

4) What other important considerations am I missing that may skew the accuracy of this project?
I go back to really knowing the market you are to research. You need to really set out the goals of the project and a good market researcher will help you design the methods to attack. You will spend about the same to have a firm do this as you will pay for about 1 year of a researcher's salary if it as big as you make it sound. Do you want internal capabilities to go back and check the markets? Your call.

Good luck - I have been hired to do this for the past 3 years and my survey finished about 2 years ago is still valid. Done externally for a reasonable price, we now use internal resources to check if things are changing. I am able to dashboard this and keep the operations leaders as well as our president aware with minimal other reporting.

John

Your commitment to really understanding your market is refreshing. As you know, most marketers would sooner "wing it" than dig in and do the hard work to quantify consumer habits, practices, attitudes, awareness, values, beliefs, and all the other things that can point to creative solutions in business.

As you go down this path, you might want to consider two preliminary mini-steps:

(1) some qualitative research to identify the issues that might be less obvious, learn the language people use to express themselves when discussing the category, and get a sense of what's most important -- from body language, facial expressions, etc.

(2) a very limited pilot test, to make sure the survey instrument works the way you want it to, can be administered effectively and efficiently, and yields the kinds of results you want.

As an engineer by training, I gravitate to quantitative studies -- as you have -- because I know there's "truth in the numbers." What I've come to learn is that there's also some "truth" in my right-brain, and the best way to really uncover the insights that make a difference is to get both halves of my brain working together.

You have clearly embarked on a challenging project. My suggestion is that you enlist the professional help sooner than later. You'll end up saving money, getting better results for your money, and probably getting it sooner/faster.

If you need a recommendation for market research professionals who can handle projects like this, let me know.

And good luck as you get into this. Let us know how it goes!