Why The Perfect Attribution Model Doesn’t Exist

As the marketing director at an ecommerce agency, I’ve met more than a few ecommerce analysts who are caught in an endless pursuit for the perfect multi-touch attribution model.

I’m here to let them — and you — in on a little secret: It doesn’t exist. Read on for four reasons chasing a perfect attribution model is a waste of resources, as well as four ways that time could be better spent on other methods of analysis.

A Fool’s Errand

1. A model is, by definition, a simplification of reality.

The purpose of an attribution model is to summarize historical performance in a way that influences decisions made on marketing efforts in the future. To serve that purpose, the model has to be digestible, concise and actionable.

How do analysts get lost in the pursuit of a perfect attribution model? It’s tempting to think that since we can measure it, we can understand it.

But have you ever looked at the top conversion paths in Google Analytics? As you get further down into the report, the number of possible paths to purchase explodes.

In the image above, there’s a path that includes 13 direct visits. In fact, this path resulted in six conversions in this particular account.

The pursuit of a perfect attribution model would assume that you would not only be able to model every permutation of a path to purchase, but you also would be able to quantify each component’s value in driving a visitor to purchase. With a nearly infinite number of options, that simply isn’t possible.

2. Your predictions are likely to be self-canceling.

In Nate Silver’s “The Signal and the Noise,” the author gives an excellent example of a self-canceling prediction.

In a world that’s becoming increasingly dependent on GPS, a huge portion of the population uses the tool to avoid delays along their routes. But if enough people are using GPS, re-routing can actually cause traffic to redirect itself. This could cause the original route to wind up being the faster option.

Creating the perfect attribution model is similar in that the action your marketing team takes on the data often winds up canceling out the prediction — or at least altering it.

This is especially true when marketers determine a particular channel is producing good results and pump too much money into it. In many marketing channels, the first dollar spent in advertising is more efficient than the last.

3. Predictions based entirely on data are inherently flawed.

The explosion of big data over the last decade brought about the misguided belief that simply “mining” the data will tell us everything we need to know. Data with no understanding of the underlying mechanics is almost useless.

For instance, let’s take another look at the conversion path I showed earlier in the article with 13 direct visits.

This could be a customer who hit a technical problem and finds herself incapable of placing an order that she would gladly have placed after one direct visit. Or maybe it’s someone so over-the-moon about your product that he keeps showing his friends, calling over anyone who will look at his monitor.

Blindly relying on quantitative data without ever giving it real-world context often leads us down the path to bad conclusions, which become the basis for even worse decisions.

4. Technology is simultaneously helping us measure and making it impossible to measure customer journeys.

We’ve come quite a way in data collection methods and analytics tagging. However, while that was happening, the mobile, tablet and smartwatch industries were all born, creating infinitely more complex customer journeys than our technology is built to understand.

Does your current attribution model consider a browser built into a car? How does that factor into a customer journey?

Desktop internet, mobile internet… It’s all just becoming one internet that consumers can interface with in a dozen different ways.

The belief that we’ll someday have a perfect attribution model must coincide with the belief that the ecommerce shopping experience will stop evolving long enough for us to catch up. I find that really doubtful.

A Better Solution

There’s been a lot of news lately about the workplace culture at Amazon. It’s difficult to separate fact from exaggeration, but it’s been reported that new employees are quizzed on the company’s leadership principles, one of which is to have a “Bias for Action.”

Truthfully, that’s a great mantra for any analyst. If you take away only one piece of advice from this post, let it be this: You should seek out data that clarifies rather than clouds and provides insights that are decisive and actionable.

1. Collect qualitative data.

If a picture is worth 1,000 words, a conversation is worth 1,000 data points. The complexities hidden in purchase path data can become very clear in a short exchange with an actual customer.

Use these techniques to get inside the minds of your visitors:

• Run exit surveys to understand why visitors might not be satisfied with their visits to your site. I’m a big fan of Qualaroo.
• Survey existing customers. Focus on getting detailed, open-ended feedback. A simple, cheap tool like Survey Monkey does the job.
• Run usability tests on your site, especially on critical functions like checkout. You can use a site like User Testing, or even just take a laptop around to co-workers who don’t use your site every day. Ask them to do a specific task, and don’t help them.
• Talk to customer service. Customer service representatives are an untapped goldmine of unstructured qualitative data.

This kind of qualitative data adds context to your raw numbers and reveals insights you’d never have access to with just quantitative measurements.

2. Run controlled experiments.

A/B and multivariate testing aren’t just for conversion optimization. Has anyone in your organization ever wondered what the real return on retargeting is?

Ever heard, “Yeah, but how many would have returned and ordered without those ads?” Here’s your answer.

Let’s say this ad is running at a $20 cost per converted click. How many of those orders would have happened anyway?

Well, that depends. Was I just on the site? Was I there a month ago? The only way to determine the real incremental value of a retargeting ad is to experiment with it.

Run an A/B test with 20 percent of your audience receiving an ad for a non-profit. They’re the control. How do those conversions compare to the 80 percent you ran your usual ads against?

This test doesn’t do anything to convince more visitors to convert, but it does tell you (and the naysayers) the incremental value of your retargeting ads. In a world of complex ecommerce journeys, that knowledge is worth its weight in gold.

3. Look at customer-level data.

Most of the discussion surrounding attribution modeling revolves around understanding the value of the marketing that led to a single purchase.

Thanks to the “Shark Tank” education this nation has involuntarily received, millions of people know CAC (cost to acquire a customer) and CLV (customer lifetime value) are critical metrics to track for any business. Despite that knowledge, many big and small business leaders still don’t know these numbers themselves.

And there’s so much beyond just knowing a basic CAC for your business. What’s your CAC by channel? Is that paid search campaign for widgets more efficient than your outbound sales team? Where do your best customers come from?

The first thing I bought on Amazon was a $10 “Harry Potter” book in Spanish in 2007. (It was a very interesting read.) So far in 2015, I’ve placed 13 orders for a total of $630.10.

What if my marketing value was measured only by my first $10 purchase?

Data analysis here gets a little trickier, since you have to leave nice out-of-the-box tools like Google Analytics to get them. Startups like Custora or RJ Metrics help companies gather these data without serious development.

Whether you go the buy or build route, you should start collecting this data now.

4. Examine incremental ROI.

One of the biggest mistakes in marketing is assuming a channel will continue running at the same efficiency after budgets are increased.

I always make it a point to examine how “new money” performs for any given channel. Did it provide the same or better ROI? Is the ROI still acceptable on the new investment?

Not only does this help you validate previous budgeting decisions, but it also helps inform your future investments.

To be clear, attribution is an important and necessary part of marketing decisions; it’s the pursuit of a perfect or even near-perfect model that leads to wasted resources and unsubstantiated marketing decisions.

Luckily, like customer journeys, there are nearly infinite ways to carve out better, more actionable insights into your business.