Keep An Infinite Memory For Your Metrics

Image via Shutterstock

Often, you want to look at your metrics over a longer time period than usual in order to identify long-term trends. This is especially true if your metrics are “jagged” and move around a lot. You have two choices:

• You could use a larger time period. If you usually watch daily metrics, switch to weekly. Or monthly. Or quarterly. This will smooth out your data and will also smooth out directional trends emerging from your data right at the time when you want to catch such changes.
• You could use an average of values of your metrics. For example, instead of weekly period, you could examine the average over the last seven days. Since the past data examined changes every day, this is often referred to as a “moving average,” since the days looked at move forward in time as you collect more and more data.

Now, you could use a simple moving average (SMA), where the weights of each day of data is given equal weight. You might even do something slightly more sophisticated by using varying weights to the Nth day, referred to as a weighted moving average (WMA). But, either solution leaves you with two issues:

1. A loss of memory: if you’re looking back N periods to calculate your average, you’ve effectively forgotten what happened more than X+1 days ago. If something important occurred any earlier than that, too bad.
2. A data-scale problem: you have to keep the data around back far enough to re-calculate the average each day forward. Not a big deal if you’re looking at shorter periods (daily, weekly, even monthly), but this can be cumbersome if you’re examining longer trends.

Enter the exponential moving average (EMA). With the EMA, the “memory” of past metrics values is never forgotten, though it is gradually given less and less weight over time. Further, since the EMA is based on what the value of your metric is today combined with what the EMA was yesterday, you don’t have to keep large amounts of past data on hand to update each day’s new values.

I’ll show you a quick example in a moment. But if you do anything with Google AdWords, you are almost certainly concerned with something that is based on an exponential moving average: the Quality Score (“QS”). As any AdWords advertiser knows, Google reports QS scores only at a keyword level, though it’s commonly believed that Google has QS numbers for AdGroups, Ads and even entire accounts.

That’s a lot of number crunching, even for Google. And you may have noticed that when Google reports your QS to you, it’s presented as a single number, not as a time series. Thus, your QS is not itself a metric, but rather a property of your account. It’s also well known that the history of your account plays a great part in determining your QS.

Sounds familiar, right? Because those are exactly the characteristics of an exponential moving average discussed above. I don’t think I’m going far out on a limb to assert that Google is using an EMA (or equivalent) to store, score and report your Quality Score.

Why not use this same technique when examining your own metrics? Here’s how the EMA is calculated.

Let’s say you want your past data to have a strong memory and thus have yesterday’s value of the EMA be pretty important when calculating today’s value. In an EMA, one uses a factor (usually called alpha) which represents a constant smoothing of the previous values of the metric. The value of alpha is anywhere between [0,1]; the higher the number, the more quickly past data is discounted.

In our example, we want to discount past data slowly, so we’ll sent alpha at, say 1/3 (33%). That’s the weight of the current data. So, normalizing, that means the weight of the past data is (1 – 1/3) = 2/3. If the value of the EMA yesterday was, say 27, and the value of today’s metric is, say, 30, then the value of the EMA today is:

EMA (today) = (2/3)*27 + (1/3)*30 = 18 + 10 = 28

If you want an EMA that responds faster to changes in trend, then set alpha higher, so that past values of the EMA are discounted faster. For example, set alpha to 80%. Now:

EMA (today) = (20%)*27 + (80%)*30 = 5.4 + 24 = 29.4

This may give you a much easier way to track important changes to your metrics while at the same time smoothing out any spurious noise in the data as it occurs. Your mileage may vary, but if Google is doing it with your Quality Score, it certainly is a trick I’d want in my bag o’ metric wizardry!