If you were running for president and a Gallup poll directly asking voters their voting intentions said you would win yet a prominent data guru's predictive model said you'd lose—would you be confident on election night?
Mitt Romney faced this situation in 2012, and he believed the favorable Gallup poll results. After all, the results came right out of voters' mouths. But as we know, the poll was wrong, and Obama won the night.
Interestingly, Romney is the former CEO of Bain & Company, the elite consulting firm that, along with Satmetrix and Fred Reichheld, co-created a simple survey and metric widely used in business to gauge customer satisfaction: Net Promoter Score (NPS). Like the Gallup poll, NPS delivers a simple, digestible metric, which gives senior corporate management the same misleading type of measure the Gallup poll gave Romney.
Let's dive into the debate surrounding NPS and examine the existing and emerging tools aiming to disrupt its dominant position.
The Popularity of NPS
Why is NPS so widely adopted?
- NPS is simple and based on the answer to a single, clear question: How likely are you to recommend this product to a friend? A respondent noting 0-6 is considered a detractor, 7-8 a passive, and 9-10 a promoter. NPS is the percent of total promoters minus the percent of total detractors.
- Likelihood to recommend a friend may not be precise, but it is at least a general gauge of customer sentiment.
- An elite consultancy, Bain introduced NPS through an elite publication, Harvard Business Review, then a few brand-name early adopters jumped on the bandwagon.
What NPS Gets Wrong
NPS is simple and popular... but NPS has several inherent flaws that make it an inconsistent, unreliable indicator of the overall state of customer sentiment and satisfaction. Flaws include...
- It's not representative. CMO found NPS only has a 10-15% response rate, with more than 85% of total customers not represented.
- Detractors are miscategorized. "Detractors" are people who are only unlikely to recommend a product; however, there is no indication they would discourage it. Consider a different metric, Word of Mouth Index (WoMI), which only considers someone to be a detractor if, when asked, he or she specifically indicates that he or she would discourage use of the product.
- Bias abounds. People tend to respond to satisfaction surveys when they are extremely unhappy or extremely happy, resulting in an over-representation of promoters and detractors.
- It's easily gamed. Environmental factors can easily influence responses, such as when and where the question is asked (e.g., the answers given immediately after the purchase could differ from the answers that would be given months later).
- It varies widely by industry. Even the creators of NPS acknowledge there's little correlation to revenue growth in some industries.