Data Danger: How The Wrong Campaign Metrics Can Break Your Mobile Strategy

Let’s start off with a story that will sound all too familiar to the data-driven marketer.

You’re tasked with improving the results of your mobile marketing campaigns. You decide to start aggressively testing the variables within the campaigns — your audience targets, offers, creatives, message timing; the list of options goes on and on.

You home in on the one variable you believe will have the biggest impact: the offer. You set up an A/B test, and you run a push notification campaign comparing the options head to head.

Good news — there’s a clear winner! Option B has double the click-through rate, and the immediate gain is an increase in the conversion rate. High fives all around — you nailed it.

Now that you have a winner, you push 100 percent of the audience into the option B track and sit back. But a week later, you start to see an increase in churn — users are leaving. You have fallen prey to “unintended consequences.”

Avoiding Unintended Consequences

Campaign metrics have led you astray and prevented your ability to see around the corners and play the long game. How can we avoid this fate?

Let’s start by deconstructing where this test went wrong and how it could have been avoided. The problem was that you were measuring the campaign’s success on isolated engagement metrics (like conversion rates) and ignoring the downstream business impact.

This is especially important when it comes to mobile, where 25 percent of apps are only used once, and unhappy users can be quick to leave. Mobile isn’t just another channel in your digital strategy; it’s the catalyst for transforming the customer experience.

If you’re measuring your mobile efforts purely on one-time engagement metrics, then you’re optimizing for the wrong thing and brewing an engagement crisis with your users.

The way to get around this is to have a control group and to have clearly defined business metrics. It’s very simple, and it’s a core part of what top marketers know they should do with every test, but it can be harder than it seems.

With the proliferation of new tools and multiple solutions making up your marketing stack, it’s hard to adhere to this best practice.

Here are some tips for making sure you’re using the right tools and insights:

1. Do Your Homework

When you’re analyzing the latest shiny tech solution, make sure you create a bulletproof RFP (request for proposal). Always ask potential vendors if use of a control group is included and how long they can compare it against test cells.

You must be sure that you can compare the control group against the exposed groups and be able to do it for an extended period after the campaign drops. That is the only way to avoid annoying users or potentially cause them to churn.

Mobile marketing is particularly ripe with solutions that don’t provide this capability, so ask, share your use cases and make sure that they can point you to another customer who has actually done this effectively with their solution.

2. Consolidate The Marketing Stack Where Possible

APIs (application programming interfaces) are great, and integration capabilities have never been stronger, but be careful what you are integrating. In particular, avoid having one tool that represents the channels to your customer and another that does the core user tracking and analysis.

Mobile interactions happen in real time, so that kind of disconnect can pose a problem. Point solutions will give you campaign metrics, but you must rely on another source for the true user impact.

Lastly, using two solutions can cut your campaign measurement off in the middle. Downstream user metrics must be connected to campaign metrics if you want to have the full picture and avoid unintended consequences.

3. Measure Campaigns Against A Broader Set Of Metrics

Get beyond campaign metrics and focus on the true lifetime value impact of your efforts. When you connect lift in conversion to future purchase behavior, and ultimately, user churn, you will get the full picture of your results, and you can double down on that test result with confidence.

Mobile has provided an unprecedented opportunity to build deeper relationships with customers across their whole experience with your business. It presents a big opportunity to stand out from the crowd, disrupt your industry and own a disproportionate share of your customer’s mindshare and interactions.

As marketers, we have so much granular data on our users, but we’re leaving growth on the table if we don’t seamlessly connect the opportunity and the data to play the long game. We’re stewards of our brand and our customers’ experience, and now is the time to avoid going astray and to take control of our marketing results.