Using Indices For Digital Analytics: Opportunities And Pitfalls

It seems as though indices are everywhere these days as the entry-level insight tool of choice for digital analytics. It’s Marketing Analytics 101. But there is still some debate as to what an index is – and often, indices can be misused. But because each individual index can be used by marketers to help understand the effect of particular attributes (for example, age or gender) on a consumer’s propensity to convert, it can be one of the most actionable, effective tools at an analysts’ disposal.

So how do you use them? In the context of digital media analytics, an index is the ratio of an attribute’s penetration in a specific audience to that attribute’s penetration in a more general audience.

Index values higher than 1 (say, 1.25 or above) suggest that a particular audience has an attribute more often than the population at large. Index values lower than 1 (say, 0.75 or less) suggest that a particular audience has an attribute less often than the population at large.

Here’s a simplified example of an index: let’s say that over the last two quarters, 10% of all internet users have been identified as Android intenders, but 20% of visitors to acmecellphones.com were identified as Android intenders. Then acmecellphones.com has twice the penetration of Android intenders as the general population, or an index of 2, indicating to a marketer that Android intenders might be profitably targeted through digital media.

Potential Pitfalls

In the real world, it’s seldom this simple. Indices can be very useful, but there are two potential pitfalls inherent to their helpfulness.

First, marketers must take care to use a sufficiently large sample (or population) before declaring that index reliable and useful. Analysts must cast a critical eye at indices based on low populations. For instance, a conversion rate for a specific audience may be an astounding 10%, but if very few people were shown the ad in the first place, the index must not be trusted.

To use indices effectively, therefore, marketers must have enough evidence to overwhelm the prior knowledge that most things are likely unrelated.

Garbage In, Garbage Out

Second, it’s vital to take into account how your population has been selected. Let’s take an example of how things can go wrong. Suppose you have a website with twice as many male visitors than female. You could construct an index of 2 and conclude that you should target men instead of women in your next media campaign. This might be the correct conclusion to draw; or it might not.

Let’s say you had just completed a digital media campaign in which 1 million men were targeted and 150,000 women were targeted. Of the 1 million men, 500 responded (for a 0.05% response rate) and 250 women responded (for a 0.17% response rate). So, even though twice as many men visited the site than women overall, men only responded half as often as women.

Simply put, indices are only as good as the questions one asks and the population one uses to create them.

In the end, it comes down to careful planning, intelligent decision-making and technical design as well as ensuring that logical business questions drive the analysis. With these in place, indices can be a very useful initial entry into the world of digital analytics.

But if neglected, indices can lead to incorrect interpretations and decisions that can have a profoundly negative impact on the performance of your digital media campaigns. That is, knowing what’s behind an index very much matters.