Back in the mid-1980’s, Baskin-Robbins introduced a line of international cream flavors, including one named Chocolate Raspberry Truffle. It was fantastic – rich, decadent, and chocolaty with ribbons of raspberry and bits of real truffle. As good as it was, you can’t find Chocolate Raspberry Truffle anymore. Why is that? No matter how good this flavor seemed, Baskin-Robbins probably knew that it was the “flavor of the day.”

Flavors come and go. It’s a fickle marketplace (except when it comes to the top two selling Baskin-Robbins flavors: Vanilla and Chocolate – some things never change).

And whether it’s ice cream or models for understanding the customer experience, the marketplace often gravitates toward some special flavor of the day. There may not be 31 flavors for managing the customer experience, but there have been quite a few. Total Quality Management. Zero Defects. Kaizen. Kano. Benchmarking. Best practices. A WOW customer experience. Customer delight. And the latest flavor, Net Promoter.

Net Promoter is rooted in sound logic, good intentions, and the allure of simplicity. Who can argue with the notion of “loyalty matters most” and you need only ask one question – the “ultimate question” – to do right by your customers and achieve higher profitability?

Who can argue with the premises of Net Promoter? Many can and some have. Let’s look at three such arguments. In each case, let’s examine the “theory” and the “practice.”

One Size Doesn’t Fit All

The Theory. According to the manual (Reichheld, Fred (2006), The Ultimate Question. HBS Press: Cambridge, MA. 98 – 100), the Net Promoter calculation is to be based on a 0 to 10 scale for recommend intention, and the calibration is supposed to be static; 0’s to 6’s are Detractors, 9’s and 10’s are Promoters, and ; 7’s and 8’s are Passives.

The Practice. This works great – except when it doesn’t.

Recent CCMC research illustrates why this static approach to defining customers can be problematic. One company we recently worked with was a Net Promoter believer who came to us in search of some guidance on how to interpret their Net Promoter score. We discovered something interesting when looked more closely at these data.

As proponents of the slogan, “there are lies, damn lies, and statistics,” we believe that no single measure of the customer experience is sufficient, nor should those measures be viewed in isolation. With this in mind, we explored the relationship between “overall satisfaction” and the various Net Promoter categories. As I note below in our second point of this blog, satisfaction can sometimes be a better predictor of future business results than a Net Promoter score.

Here’s what we saw:

• Satisfaction levels for those providing recommend scores of 5 and 6 were virtually identical to those with recommend scores of 7.
• Those giving 8 ratings to the recommend metric had the same satisfaction levels as those recommending at the 9 or 10 level.

Essentially, a meaningful percentage of Detractors should have been designated as Passives, while a group that should have been characterized as Promoters were instead labeled as Passives. The upshot is that the Net Promoter score our client calculated was likely too low.

This example reflects the flawed assumption of a static definition of Detractors, Passives, and Promoters. This designation is more client- and situation-specific. The designation of Detractors, Passives, and Promoters is an empirical question that should be answered by calibrating results against their specific customers.

Net Promoter: It May Not Do What It’s Intended To Do

The Theory. Net Promoter makes a number of claims about the linkage between better Net Promoter scores and improved financial outcomes (Reichheld, Fred (2006), The Ultimate Question. HBS Press: Cambridge, MA. 41 – 44).

The Practice. Let’s look under the hood. One recent study found that the claimed association between the Net Promoter score and future business performance is tenuous, at best.

In their 2006 study, Professors Neil Morgan and Lopo Leotte do Rego examined the impact of six customer feedback metrics on future business performance. Future business performance was defined in terms of such measures as sales growth, total shareholder return, gross margin, and market share.

The six customer feedback metrics included: Net Promoter, average satisfaction scores, top two box satisfaction scores (e.g,. Very & Somewhat Satisfied), average repurchase intention scores, word of mouth rates, and customer complaint rates.

The results – based on sound social science and relying on a host of sophisticated statistical modeling techniques – were alarming if you’re a Net Promoter fan. Among all six of the customer feedback metrics used, the Net Promoter score ranked last with the weakest predictor of future business performance.

If you trust the results of this study (and others not referenced here), it may be worth rethinking hitching your wagon to Net Prompter.

Net Promoter: Accentuating The Risk Of Gut Feel & Intuition

The Theory. The Net Promoter bible contends that one need ask only the ultimate question (Reichheld, Fred (2006), The Ultimate Question. HBS Press: Cambridge, MA. 96 – 98). The author of Net Promoter implicitly theorizes that managers are smart enough to take this single data point and then figure out how to drive better performance.

The Practice. While the optimist in me would be inclined to say “Yes!,” my pragmatic instincts tell me “No.”

For the sake of argument, let’s make four assumptions:

• Your organization truly believes in the power of Net Promoter
• The theoretical static calibration works perfectly – just as it’s described in the book
• Your organization either doesn’t believe or doesn’t care about the validation studies that refute the association between the Net Promoter score and business results
• Your organization achieves a score on the lower end of the spectrum, perhaps 3% (like Travelers Insurance) or -9% (like CitiGroup) or -16% (like US Airways)

As a leader in your organization, what should you do?

Given the Net Promoter tenet that one should “rely on the ultimate question,” you have two options.

The first option is to succumb to “paralysis by absence of analysis;” you do nothing because determining the most prudent course of action is an enigma.

The second option is to use your “gut” to make changes; apply what you see, feel, and hear about the customer experience and go with it.

Neither alternative seems especially attractive to me. The first won’t work in most companies (I say “most” because there are still plenty of companies that measure but don’t take action). The second is akin to playing the slots, buying a lottery ticket, or betting against the spread; there’s an abundance of unknown risk tied to the outcome.

Unfortunately, many companies today find themselves in this predicament. We recently collaborated with one such company. The organization, after many years of doing traditional research to truly understand the customer experience and vet priorities for improvement, switched to Net Promoter. We were told that there was no flexibility. Executives were mesmerized by the simplicity of the approach. Each division within the organization was held accountable for improving the Net Promoter score. Not surprisingly, we got a call from the manager with a lower score (ironically, the legacy data suggested that this division was doing better than was indicated by Net Promoter results). Tasked with improving performance he asked (and had a hard time answering), “What can I do?” Was he a bad manager? I don’t think so. I’d ask the same question.

There is a third alternative. Conduct meaningful survey research to identify the key drivers of customer satisfaction and loyalty. And it’s the approach that many companies – including the one I’ve just described – have turned to in their disappointment with the lack of actionable data endemic to the Net Promoter approach.

Where Do We Go From Here?

If the objective is to minimize the risk associated with investing finite resources in those actions that yield the best customer experience ROI, Net Promoter comes up short. This conclusion begs the question – if Net Promoter isn’t the way to go, what is? Stay tuned for my next blog entry…