Company: Major League Baseball
Contact: Dan Derian, MLB Senior Director of Research
Location: Location: New York, NY
Industry: Professional Sports, B2C
Annual revenue: $6,000,000,000
Number of employees: 250
Quick Read:
Each year, when Major League Baseball's postseason culminates with the World Series in October, all eyes are on the MLB—or are they? With most teams out of the running, do those teams' fans actually continue to watch? Are the time and resources invested in marketing such high-profile events worthwhile, or even in tune with fans' true passion for the game?
Such insight used to be financially infeasible for the MLB's Senior Director of Research Dan Derian. But the introduction of an online advisory panel in late 2006—dubbed the "MLB Fans at Bat" and made up of avid supporters from all over the country—has allowed the organization to successfully double its research efforts.
Moreover, it helped make the 2007 postseason far less ambiguous, answering not only which fans tuned in but also whether (1) Dane Cook was a good spokesperson; (2) the message was communicated clearly; (3) and the overall creative material was on target.
The Challenge:
Only in the last decade or so has the MLB turned to field research to validate and shape its business decisions. Once it was initiated, its value was quickly realized. But the time and costs associated with primary research restricted the organization from performing the breadth of fieldwork it desired.
"We wanted to be in touch with our consumers as often as we could, but we were limited somewhat by budgets and costs," recalled Dan Derian, senior director of research.
The MLB had to find a more cost-effective approach to research.
The Campaign:
On the recommendation of his National Hockey League counterpart, Derian contacted Vision Critical, a Canada-based firm that specializes in Web-based panel management tools. In August 2006, the "MLB Fans at Bat" online customer advisory panel began to be implemented.
Close to 20,000 panelists were recruited from the MLB.com registered user database via an introductory email message:
"You have been selected to join a newly created Major League Baseball advisory panel made up entirely of fans just like you. By signing up, you will be asked to give us your opinion on a wide variety of topics important to MLB. Major League Baseball fans are some of the smartest and most vocal fans in all of professional sports, so here is your chance to talk to us and help make a difference. We are listening."
Rather than rely on incentives to encourage participation, MLB decided it could leverage the true passion of its fans and positioned the panel as an "opportunity to share their thoughts, to stand up and be counted," said Derian.
Willing participants were asked to complete a lengthy profiling questionnaire that enabled the MLB to screen them according to demographics and team affinity and easily determine subsets within the sample, as needed. This data was later appended to all supplemental surveys, drawing a more complete picture each time research was performed.
The first survey was conducted in November 2006, and since then panelists have been sent a new questionnaire once a month on average, creating an ongoing dialogue with the fans without overwhelming them.
"We can segment a sample of this size so that we can still do a survey every couple of days if we want to without inundating [the panelists] too often, and the results are still representative," said Derian. Responses tend to indicate a trend after the first several hundred replies, he said.
Questions typically cover a range of topics, from feedback about a particular ad or the fit of a potential sponsor with the MLB brand, to which team has improved most during the off-season or whom they'd like to see participate in the Home Run Derby. "We have to be conscious of the types of surveys we're sending out. We are often exploring very specific business issues...but it is important to balance our business objectives with the issues [our panelists] find interesting from a fan perspective," said Derian.
As part of its licensing agreement with Vision Critical, MLB can send out as many surveys as it desires, to the entire panel or any segment, at any time, by independently copying and pasting its questions into Vision Critical's "Panel+" application. It allows such questions to be structured as single select, multi-select, rank, or open-ended, and surveys can further include images and video, which have been particularly effective for testing logos, uniforms, and television ads.
Reporting through the Panel+ system records every response and connects new data with all previously generated findings. Data can then be manipulated in the online system and exported to MLB's own Excel, PowerPoint or PDF programs.
The Results:
- Great response rates: Whether because of the new medium, the perceived privilege of participating, or simply the fervor of its fans, the MLB's surveys have had high response rates: on average, 50% for surveys completed to date—7% higher than the average for Vision Critical client panels. About 44% of the "MLB Fans at Bat" panelists participate at least 60% of the time.
- More research: The ability to send out more surveys has permitted the MLB to double its field research efforts and cover a wider range of topics. "The value has been significant in terms of what we've gotten from replicating studies we've done in past, in addition to those we might not have done due to the cost implications previously," said Derian. "We've done at least twice as many surveys as we would have done had we needed to go 'out of house,'" he added. Moreover, MLB has used the technology internally, and found it to be a great tool for surveying staff and evaluating corporate initiatives.
- Faster turnaround: With the panel in-house, MLB can initiate a new survey the very day it decides to do so, and results arrive within hours rather than weeks. On average, MLB receives 80-85% of its feedback within the first 36 hours.
- A well-qualified and readily available sample: By recruiting from the MLB.com database, MLB saved the costs of purchasing samples without forfeiting its requisite for devoted baseball fans. And because these panelists opted in, they appear more willing to participate. To "keep the panel fresh" and make room for new participants, MLB regularly purges "non-responders," who average 12% per survey.
Lessons Learned:
- A direct link is preferable: Managing the panel in-house has saved both time and money, given MLB more control over its research, and developed a dialogue with fans that allows the organization to keep tabs on market preferences.
- Balance your sample's needs with your own: Panelists eager to participate can quickly become alienated if not stimulated with questions they find interesting. Accordingly, MLB designs its surveys to include game-related topics that speak to the passion of these fans and help compensate for questions related to MLB business objectives.
- Refrain from inundating your sample: To nurture ongoing participation, Derian suggests soliciting each panelist once a month at most. By recruiting a large sample up front, he enabled his organization to conduct considerable research and achieve representative results without overwhelming panelists.
Related Links:
- MLB (www.MLB.com)
- Vision Critical (www.VisionCritical.com)
