Test-and-learn steps for agencies

We have seen a rapid expansion of entertainment and news options for consumers over the past few years. This explosion of options has fractured the traditional marketing mix, presenting a myriad of new choices for clients and the agencies who support them.

Leading agencies are always seeking opportunities to drive their clients’ business forward. Understandably, however, agency staff and account managers are leery of failure: Nobody wants to tell a client something they tried didn’t work. So, how do agencies prepare their clients for a robust test-and-learn approach?

In this article, I’ll outline a few steps that can improve the chance of finding the next great idea for clients.

Step 1: Develop a clear hypothesis for client success

Agencies need to have a clear understanding of their client’s problem and present well-informed ideas about potential solutions. The more clearly the agency and client define the challenge together, the easier it will be to design tests and interpret results.

At this stage, it’s essential to be specific. Does your client want consumer consideration? Are they looking to increase consumer loyalty? Clear goals help define a testing journey that will lead to more definitive results.

Step 2: Focus on ideas with the highest potential

We all know there are bad ideas! In fact, many of us have probably been burned by one or two of them. As marketing professionals, we’re generally looking for ideas that will generate an above-average ROI for our clients. No client is looking to spend more for website visits, consumer purchases or in-store traffic. Agencies seeking to find more of these good ideas will have a simple way to identify concepts with a higher-than-average chance of increasing ROI.

One way we use to identify better options is running a quick back-of-the-napkin spend-to-impact response function (SIRF) analysis. SIRFs are the building blocks one can use to drive increased return on investment.

Here are the numbers needed:

1. Spend: Cost of program (Consider costs to produce separate from distribution).
2. Impact: Estimated sales lift (Calculate potential loss of effects with increased frequency).

Agencies can do this analysis very quickly. Just compare results to average ROI and see if you have a shot at increasing marketing productivity.

If the ROI doesn’t seem to work with this simple analysis, move on. You want to focus your attention on the ideas that have the most merit.

Step 3: Figure out if the idea has staying power

If the idea seems to have merit, you’ll need to figure out if the approach will scale. Is the idea going to end up being a one-and-done, or can it be repeated? Does it have an impact on a niche of potential customers, or will a more extensive audience respond?

In our simple approach, we did a simple analysis of 1,000 people. We only included the cost of media and left out any potential diminishing returns. But in any test of a marketing effort, we have to remember that the first impression is probably going to give us the most significant impact. Subsequent exposure will have less impact. The ROI will decline as the program reaches more people or reaches the same people multiple times.

Here’s a simple approach to see if an idea can scale.

In this example, the client receives +6 pts from first impression, another +3 from the second impression, and +1 from the third impression. You can plot this on a simple formula with a reach and frequency curve.

Step 4: Identify the benefits of reach versus frequency

There are two factors in every marketing effort that one needs to consider: reach and frequency. Seeing how these will impact the successful idea is a key to identifying winners. Here are two scenarios:

In the first, the client’s message reaches a large number of people, but the majority see the campaign only once. This example gives your client more reach but less frequency.

In the second, fewer people see the client’s message, but they see it more often. This example would give your client less reach but more frequency.

We already discussed that fact that most of the immediate impact and ROI will come from the low-hanging fruit: consumers already inclined to respond. Subsequent efforts (more frequency) will of necessity be reaching harder-to-sell, less responsive consumers. ROI will decline as frequency grows.

This quick sensitivity test can help the agency better identify efforts more likely to drive a client’s ROI and improve business results.

Step 5: Identify meaningful success metrics early on

If you’ve made it this far, you’re probably ready to move forward and present the idea to your client. Now is an excellent time to reassess your hypothesis and make sure you’re collecting the data that will help you assess the performance of the idea.

For example, one of our clients was testing new messaging in digital advertising. When the data was initially analyzed, it appeared that the new ads were a home run: Their online traffic was experiencing huge gains. However, digging a little deeper into the metric that mattered (conversions), it was apparent that most of these new visitors were leaving without taking any meaningful action. They were “empty calories.”

Final thoughts

I am a big believer in test-and-learns. I’ve baked them into my company’s culture, and they’re a part of every one of our client engagements. The process we’ve outlined above can help you unlock more potential for your clients through tests-and-learns. It will also lead to a database of institutional knowledge that becomes a strategic asset for your agency.

We recommend every client dedicate at least a portion of their agency’s time and their budget to challenging existing plans and uncovering the next big idea.