Metric-lust: how well can you really measure marketing ROI?

Published on July 14, 2011 by James Archer

*This article was originally published on Forty.co. Forty is now Crowd Favorite.

As technology has given our industry the ability to measure more things faster than ever, there’s been a steady trend towards having metrics serve as the basis for design and creative work, with the idea that this will ensure return on investment (ROI) for marketing efforts.

It’s safe. It’s predictable. It’s simple. It’s easy to show your boss. It’s easy to show the client. It makes everyone look and feel good.

And it’s wrong.

The industry currently treats metrics-driven marketing as if it’s the inevitable way of the future, and that’s actually pretty sad. What I’m about to say challenges some deeply-held assumptions in the marketing and advertising industry, and I’m anticipating some passionate arguments against it.

Metrics-driven marketing is a gross over-simplification of sentiment and behavior.

“Marketing by numbers” can give you incremental improvements, but it generally can’t revolutionize a market, give you a radical edge over competitors, or generate an evangelizing fanbase.

Humans are complex, and groups of humans are exponentially more complex. However, agencies continue to promote (and their clients continue to be seduced by) the myth that a market’s sentiment and behavior can be accurately reduced to numbers in a spreadsheet.

Data is useful. It’s a piece of the puzzle. But it’s not enough to make good marketing decisions.

Client and agency metric-lust is understandable, of course. When you’re dealing with something as intricate and unpredictable as marketing, the notion of reducing it to a number is immensely appealing. But the data doesn’t accurately represent reality, and there are a lot of businesses making a lot of bad decisions as a result.

Marketing is one of the most complex of all human endeavors.

Marketing comprises hundreds of different disciplines — psychology, sociology, literature, anthropology, linguistics, art, etc. — all operating simultaneously and influencing each other in complicated ways.

We’ve been trying to solve the problem of moving people to action for millennia. (Aristotle’s principles for persuading groups can be found in the Rhetoric, for example, which dates from the 4th century BC.) And I’ll let you in on a secret: as an industry, we’re not even close to having it figured out.

We have some broad ideas, and there are some people with impeccable instincts for the work. But generally speaking, nobody really knows how it all works — and they certainly haven’t boiled it down to accurate numbers yet.

I genuinely wish we had a number that told us how well we’re doing. Our clients would love to match a branding strategy directly to an increase in profits. In most cases, though, it’s just not possible to accurately reduce the sum of the effects to an easy-to-handle number. Even revenue can be a poor indicator of marketing effectiveness, because it doesn’t account for long-term effects. (Brand-building won’t show you major results for months or even years.)

Good marketing can be measured. GREAT marketing often can’t.

Think about yourself for a moment. Think about the way things influence you. Think about the brands you truly love and the brands that influence your behavior.

Are you so predictable and reactionary that an Account Executive somewhere can plug you into a spreadsheet as “proof” their campaign worked? Or are your decisions (and the way you act on them) more complicated than that?

Let’s say you bought an iPhone yesterday. Apple, a company with a long-term marketing perspective, is insightful enough not to attribute that action to, say, a single ad campaign. They understand your feelings about the iPhone now are the sum of years of cultural and marketing influences, not a direct reaction to the commercial you saw last night.

“They [Apple] understand your feelings about the iPhone now are the sum of years of cultural and marketing influences, not a direct reaction to the commercial you saw last night.”

Marketing by numbers will buy you some success. It’s great for simplistic and short-term results, and playing those numbers can generate a bit more revenue over time. But you won’t ignite the public’s imagination using a metrics-driven marketing approach, because you’ll ignore all of those cumulative effects essential to turning a brand into a cultural institution.

Perceived credibility is seductive, even when we know it’s a lie.

You already know numbers are deceptive. You’ve heard the quote about “lies, damned lies, and statistics.” You know how easy it is to twist study results to fit a conclusion. You roll your eyes at the idea of “2.5 children.”

Yet when confronted with a situation as intense (because money is involved) and complicated (because people are involved) as marketing, we all tend to fall back to the spreadsheets for comfort.

Why? Because we hate uncertainty. If you spend $30,000 on a new website, you want to know it’s working, so you look for numbers that tell you something significant: hits, conversions, goal funnels, heatmaps, etc.

Don’t get me wrong, those are important numbers to look at — as long as you don’t confuse them with reality. You should measure everything possible, but you should also take those measurements for what they are: imperfect and partial views of a complex situation.

The real danger is when you begin to treat those numbers as if they’re reality, and you turn your attention to making numbers go up or down instead of focusing on genuinely making a name for your business.

Numbers turn marketing into a video game (and that’s a bad thing).

Agencies love it when clients succumb to metric-lust, because it makes their jobs a lot easier. No longer do they have to seed a revolution, reform a society, or plant an ideavirus. They can simply make numbers go up and down, which is relatively easy.

Video games appeal for the same reason. If you play Call of Duty, you don’t have to worry about the infinite number of variables found in genuine warfare: danger, guilt, ambiguity, politics, culture, relationships, regret, intelligence, fear, etc. All you have to do is put bullets in bad guys. At the end of the round, you’re assigned simple numbers informing you how well you did. It’s fun, but it doesn’t even approximate reality.

Anyone can play a numbers game. I can do something today that will make your marketing metrics go bonkers tomorrow, making me look awesome and making you feel awesome. It’s hard to shake that fantasy, but your business won’t actually budge in the long term, because I was just playing for the numbers — not for reality.

If your marketing feels clean, straightforward, and measurable, you should feel worried right about now. Are you really marketing, or are you just playing the marketing video game?

The only thing worse than no data is bad data.

Marketing metrics can give you glimpses, but they can’t give you the whole picture. Your “conversion funnel” may help you identify potential problems with customer experience flow, but it won’t tell you your embarrassing logo is making people lose faith in your company. There’s no number for that.
People measure what’s easy to measure, but that number is seldom what’s most central to building a brand and reputation in a market.

You may feel “some data is better than no data,” but that’s actually incorrect in many situations. Sometimes understanding you don’t have significant data (while it’s a disconcerting feeling) is essential to making an appropriate strategic decision.

Sometimes “I don’t know” is the right answer, and you need to move forward with uncertainty.

People feel before they think.

Neuroscientists have demonstrated repeatedly that emotion and associations come before rational thought, dramatically influencing the conclusions people make.

A study published in 2004 found that, in a blind taste test, half the subjects asked choose Pepsi over Coke. However, when subjects were told what brands they were drinking, their brain activity significantly changed (with increased activity in the areas related to cognition and memory), and 75% reported that Coke tasted better.

Their tastes actually changed because of the associations they had with the brands. That’s powerful marketing, and that’s something spreadsheets won’t tell you how to do.

Emotions are the priority in first-world countries.

You’re probably familiar with Maslow’s hierarchy of needs. At the bottom of the pyramid are the essentials (survival and safety), with which people are least likely to be swayed by emotion. If you’re struggling to meet those needs, you’ll be in a very practical and rational mindset.

However, most of your customers probably don’t fall into that group. They’ve probably moved on to the higher levels of the hierarchy:

Love & belonging: friendship, family, sexuality, affiliation, brand tribes, etc.
Esteem: self-confidence, achievement, respect of others, etc.
Self-actualization: purpose, creativity, meaning, authenticity, vitality, etc.

In other words, our higher needs are affiliation, aspiration, and identity.

To build a brand that grows, thrives, and endures, you need to tap into the top tiers of this hierarchy of needs. That’s what people are craving, and their driving purpose is to satisfy those needs. If you’re focused entirely on incremental modifications to drive numbers up, you’re missing massive opportunities to turn your brand into a vehicle that fulfills those needs.

Humans understand humans best.

The great under-appreciated fact of marketing is the human brain is wired to understand human sentiment and behavior. It does a really good job at it. Some of the greatest design and marketing work of all time has been done in the face of numbers that didn’t seem to make sense, because someone knew it would work and trusted their instincts.

For example, when Don Chadwick and Bill Stumpf from Herman Miller designed the Aeron chair, they did essentially the opposite of what the metrics told them. When the market metrics demonstrated a demand for soft, attractive, and elegant chairs, they built a hard, ugly, and confusing one — and it became one of the most coveted chairs in history.

Human instinct isn’t always right, but it’s right way more often than we give it credit for. The brain can process massive amounts of information and extrapolate subtle (and sometimes surprising) conclusions that can form the basis of a marketing strategy.

Compared to the marketing-by-numbers approach (which is about as realistic as a video game), the well-informed instincts of a design/marketing/advertising professional can produce remarkably innovative and effective campaigns that leapfrog over competitors. Metrics should be part of the information gathered, but should never be the focus and foundation of the campaign itself.

Get human, and think past the numbers.

The need to look beyond metrics isn’t just touchy-feeling and creative gibberish. Human behavior is based primarily on feelings and emotions, and only secondarily on logic and analysis. Failing to account for that in your marketing strategy (or believing you can reduce it to numbers) puts you at risk of spending large amounts of money on something with very little basis in reality.

It’s a tough sell, of course. It’s hard to throw away the metrics crutch and stand on your own two feet, trusting your gut over the spreadsheet. But that’s where the truly market-changing results can be found, and where you can build a client base that truly loves and connects with your brand.