by Kevin Horne
- View article on one page
- Page 1 2
It's hard to believe how different it was, just a mere 15 years ago, to conduct secondary market research. There was no Internet (commercial at least), no Yahoo portal, no Google search, no Web-accessible databases to tap. Almost every effort required a phone call, a trip to the library, a subscription to a third-party source, a read-through of hardcopy reference material.
How times have changed.
But, not always for the better. The seemingly bottomless pit of content that makes up today's Web poses some distinct challenges to marketers looking for precise, credible facts on which to build a strategy.
This article points out some of the online research traps—which can put your marketing strategy atop the proverbial house of cards.
Misleading Definitions
It is not uncommon for a data source to use imprecise terminology. My favorite example is the oft-quoted number of cell phone users in the US. You constantly see or hear 255 million, since that number is prominently displayed on the home page "ticker" of the wireless trade association CTIA.org. It seemed high to me, and when I contacted CTIA's head researcher, he admitted the number is for subscriptions, not subscribers.
When I later saw the 255 million number used, once again, in The New York Times April 13 "Week in Review" section, I requested a correction. The Times did its own research and two weeks later published a retraction, with a new number of 226 million.
Other data definitions to double-check include households vs. individuals, visits vs. visitors, total population vs. Internet users. These types of mistakes can really throw off your work in sizing market opportunities.
Bad Math
Most mistakes in this category seem to come from analysts' providing interpretation of data, comparing one set of numbers to another.
For example, research reports frequently reference market growth. Be careful: For example, an analysis might say that a company went from 10% to 11% market share, achieving 10% growth (1 divide by 10). Misleading, and easily misinterpreted by your researcher.
A more dangerous form is where interpretations are developed on "scaled" qualitative information. For example, if 60% of people rated factor A as a "5," and 30% rated factor B as a "5," the analyst might conclude people are twice as likely to choose A over B. Nonsense. Don't be swayed by this kind of math. (The scale may be using numbers, but it's not quantitative; it could just as easily be using "OK, good, better, awesome, and way cool...")
- Page 1 2
- View article on one page
Comments
by Dan Soschin Tue Jul 22, 2008
My suggestion to those of you publishing results from your research is to also ALWAYS publish how the data was obtained and the full data set... this is especially true if your data shows something that is hard to believe.
If you were shocked by the data, then chances are, so too will your audience. Site your sources and methodology so you can add credibility to your data!
by J Geibel Wed Jul 23, 2008
The article is a good start but the initial premise "marketers looking for precise, credible facts on which to build a strategy." is a bit off.
Strategies aren't always "built" on facts - strategies are developed as a byproduct of a market vision, and market research is used to validate or shape the underlying assumptions.
Unfortunately (or fortunately, depending on how you look at it), marketing isn't accounting and often there aren't "facts" to support key assumptions in a marketing strategy - for example - there weren't any "facts" to support the use of personal computers when Apple introduced theirs in the late 70's. Nor were there any "facts" to support the use of cell phones when they were introduced in the mid-80's.
What this means is that the unknowns in a strategy have to be clearly highlighted and monitored as the go-to-market strategy is implemented. Those assumptions will be either validated or disproved in the initial marketing efforts - and the strategy has to be adjusted in real time. That's why many market introductions are so tough.
All this being said, the author brings out some good points about checking (and cross-checking) any "facts" that are published. There is a tremendous amount of misinformation and disinformation on the Internet and even published by major media (the fact checkers were laid off years ago). However, the Internet also makes cross-checking easier. My rule of thumb is to triangulate - I look for three sources that validate the "fact" or figure - and not ones that merely reference each other.