The post Beyond heuristics: Algorithmic multi-channel attribution appeared first on MarTech.

]]>Marketing attribution continues to provoke many discussions, theories and debates these days. As stated in the intro to Christi Olson’s recent Search Engine Land column, “Proper attribution modeling is one of the biggest challenges facing marketers today.”

Along with difficulties caused by holes in data (such as connecting user journeys across devices), the oversimplification of traditional first- and last-click models is highlighted.

These models fall into a class of “heuristic” rules. A heuristic, by its nature, is a simplification of problem to more of a “rule of thumb,” removing complexity in favor of a quick analysis. In the case of attribution modeling, this means assigning values across positions in the chain, **regardless of actual impact on the completion of a sale.**

The step beyond this is to algorithmic attribution — **complete analysis of the available data to determine the true impact of a given touch point on conversions.** Rather than “shortcutting” and applying a blanket position or time-decay rule, algorithmic attribution involves having a custom model and weightings for each touch point based on your own user dynamics.

Alongside a truer picture of channel value, the deeper understanding provides a starting point in progressing away from descriptive analytics towards the realm of predictive and prescriptive analytics. See Garnter’s useful visualization of the types of analytics and their value:

Getting this fuller understanding of channel influence is a key step in moving towards both predictive and prescriptive analytics (as opposed to descriptive, which merely tells us what happened historically).

This is something you may have encountered if you are a Google 360 (or formerly Adometry) customer through their “data-driven attribution” feature.

Let me illustrate by showing an example approach, which uses a Markov model.

In simplest terms, Markov chains are based on modeling the probabilities of transitioning from one state to another. An example would be forecasting tomorrow’s weather: if it’s sunny today (current state), what are the probable weather outcomes tomorrow (future state)? We can visualize some probabilities in the format below:

If it’s sunny today, there’s a 10 percent chance of transitioning to a “rainy” state tomorrow. If it’s rainy today, 50 percent of the time it will be sunny tomorrow.

Good question! Let me explain:

We can consider our conversion journey as a series of states based on the referring channel. In other words, we have various states and paths such as:

As with the weather, we can crunch the data we have on these paths (easily accessible in Google Analytics, AdWords and so on) and get some probabilities for transitioning between the channels, towards a successful conversion.

Our first step is to break this down into individual “transitions” so that we can do a simple count of occurrences. Then, based on number of instances of a transition vs. possible transitions, we can work out the individual probabilities:

Plotting this out into a graph, as per the weather example, gives us the following:

For example, we can see that if the last visit was from the organic search channel, there is a 40-percent likelihood that this is followed by another visit via this channel, 20 percent that the user is never seen again (the Null node), 20 percent that the next engagement is via PPC and 20 percent that they convert at this stage.

But we’ve still not cleared up how this helps with the attribution problem. Well, the application is based on assessing the impact of removing a specific node (or “channel”) and using our remaining probabilities to assess the drop in completed conversions in the absence of this touch point.

We are essentially weighting on the basis of the “removal effect.” If the CPC channel is null, our channel interactions are simplified to the following:

The “removal effect” of dropping the CPC touch points is dropping one of our two measured conversions.

If we do the same process for the organic channel, our effect is slightly different. Neither of the conversions actually occurs due to the impact of the organic channel on the conversion completed by the CPC channel — so the “removal effect” is the reduction to zero conversions:

We can then just calculate out a weighted score on the basis of the relative removal effect across the channels, thus reflecting the greater contribution of the organic channel:

Obviously, it’s a non-trivial problem once you step beyond “toy” examples such as above, but the hope is that understanding the inside of this “black box” is useful for anyone working (or considering working) with an algorithmic approach.

Additionally, we didn’t address the inherent problems of getting “clean” channel data as highlighted by Olson (cross-device and so on), but hopefully, this served as a good introduction to the concept of algorithmic vs. heuristic models.

**A note on implementation:**

Much of my analysis would be done in R, and there is a great package, ChannelAttribution, which does much of the heavy lifting. And incredibly usefully, there is also a web-based version with some dummy data to illustrate how this approach compares to the standard heuristic models. The format of data required also means it’s possible to upload pretty standard outputs from Google Analytics.

The post Beyond heuristics: Algorithmic multi-channel attribution appeared first on MarTech.

]]>The post Machine learning for large-scale SEM accounts appeared first on MarTech.

]]>A key challenge when working on what we could term “large-scale” PPC accounts is efficiency. There will always be more that you could do if given an unlimited amount of time to build out and optimize an AdWords campaign; therefore, the trick is managing priorities and being efficient with your time.

In this post, I will talk about how concepts from machine learning could potentially be applied to help with the efficiency part. I’ll use keyword categorization as an example.

To paraphrase Steve Jobs, a computer is like “a bicycle for the mind.” The generally understood meaning of this statement is that, in the same way a bike can increase the efficiency of human-powered locomotion, computers can increase human mental productivity and output.

With the existentialism out of the way, let’s get to something tangible — We’ll explore here how relevant/valuable it could be to try and automate the process of placing new key phrases into an existing campaign.

As a definition of sorts that ties into our objectives, let’s consider the following to be true:

**Machine learning** is a method used to devise models/algorithms that allow prediction. These models allow users to produce **reliable, repeatable decisions** and results by learning from historical relationships and trends in data.

The benefit of “reliable, repeatable decisions” is the exact value that we’re interested in achieving in this case.

[Read the full article on Search Engine Land.]

The post Machine learning for large-scale SEM accounts appeared first on MarTech.

]]>The post Three golden rules for forecasting appeared first on MarTech.

]]>Ah, forecasting — it’s a task I personally enjoy, but I know it’s not a universally loved process. That said, forecasting demand is an inevitability for anyone working in any form of marketing.

If you or a client is going to invest money, you’ll want to have at least some idea of what your ROI will be.

If you are setting targets for the year/month, it helps if you are well-informed, rather than taking a stab in the dark (or worse, just setting your ideal figure blindly).

I didn’t want to provide another practical “how-to” guide, but rather give a conceptual view of three golden rules to help make sure your process of forecasting is as valuable and actionable as possible.

There are lots of posts and resources on the best methods and data sources to use for forecasting, so I won’t cover that here.

[blockquote cite=”Nate Silver”]We must become more comfortable with probability and uncertainty.[/blockquote]

By its nature, forecasting is a step into the unknown. There is very little chance that your forecast will be 100 percent correct. Embrace this fact, and you’ll find that it’s both educational and rewarding to start thinking about your forecast as a “range” of possibilities.

Here’s why I say educational: If you are forced to consider how much you don’t know, how much your metrics can vary, what factors you are reliant upon and so on, then you are forced to understand more about the dynamics of your market. This immersion in the influences is wholly valuable in terms of the knowledge it will provide you.

You may build in your ranges by **confidence intervals** (statistically derived measures of uncertainty) or by simple **scenarios** (e.g., “What does it look like *if* we manage to increase conversion rate by 0.5 percent?” and “What does it look like *if* conversion decreases by 0.5 percent?”).

The benefit of the latter example is that you are now armed with a sketch of the playing field for many eventualities (“Argghhh, conversion rate has dropped, what does this mean in the context of the target?” **pulls out scenario forecast**). Of course, it doesn’t solve your problem, but it puts you in a good position of knowing what the impact will be, the size of the tasks at hand and a tangible scope of traffic/revenue increase to chase.

[blockquote cite=”Jonathan Swift”]*It is useless to attempt to reason a man out of a thing he was never reasoned into.*[/blockquote]

Knowingly or not, you will be making many, many assumptions when forecasting.

For example, there are lots of step-by-step tutorials on how to grab some keyword data from Google, crunch the numbers and voila, there’s your forecast. This approach is fine to follow, but two huge assumptions are baked in:

- The Keyword Planner data and click-through rate data you’ve used are based on truth.
- Things will continue as they are in future.

My advice here is not to change approach or try to guard against it, but simply to acknowledge that you’ve made these assumptions, and annotate your forecast to this effect.

Making clear the assumptions you’ve made will be useful for both yourself and others for future reference. From the get-go, it embellishes the raw data with useful context (e.g., “This assumes that the temperature during key months follows the pattern of the last five years.”). This ensures that everyone who has an interest in your forecast is under no false illusions when it comes to the presence of uncertainty.

[blockquote cite=”Arthur C. Nielsen”]Watch every detail that affects the accuracy of your work.[/blockquote]

There is a fine line when it comes to revisiting forecasts — I always try and warn against the “re-forecast because we’re not meeting target” trap. Not that it shouldn’t be done, just that jumping straight into this means you’ll pass over opportunities to draw out information by comparing the forecast data with reality.

Revisiting a forecast can be highly valuable when it comes to uncovering operational or strategic insight. To give a couple of examples:

**Accuracy.**Measuring how accurate your model is in the face of reality can tell you much. Did you underestimate the impact of an important factor? Is variability much higher than expected? Is the method used totally unreliable? Introspection of the approach and results is one of the single best ways to improve future forecasting performance. (A number of best-practice methods, such as Mean Percentage Error, are outside the scope of the article, so you can read more here.)**Assessing underlying causes.**Assuming you have confidence in your model, investigating reasons that you are above/below your expected range can very useful. Say you decided to re-invest a greater percentage of your budget in programmatic display than you have done historically, and overall your revenue increased vs. prediction. Seeing that the ROI improved is one thing, but seeing the overall impact in terms of beating expectations is a much more powerful message.

Hopefully, what this post has lacked in practicality, it’s made up for in food for thought. I’ll wrap up by saying that forecasting will always be an interesting blend of science and judgement.

Embracing these in equal measures should allow you to create more useful forecasts for yourself and to continue getting value beyond the initial number-crunching which forms the basis.

The post Three golden rules for forecasting appeared first on MarTech.

]]>The post Visual analysis of AdWords data: a primer appeared first on MarTech.

]]>Data visualization is one of the most powerful tools available if you want to explore and understand your data, whether it’s on a small scale or at a scale which qualifies it as “big data.”

In this post, I wanted to run through some of the fundamental elements of data visualization and illustrate why these concepts start to reveal insight once combined.

I’ll use a very simple set of data with some fairly logical conclusions in order to focus on the effect of different techniques, avoiding adding any unnecessary complexity.

For the purpose of this post, let’s consider a scatter plot approach for a modest set of AdWords keyword data. My fictional dataset consists of data for ~700 keywords for a period of one month, with fields reflecting cost, clicks, conversion and revenue metrics.

As a starting point, let’s plot the cost per click (CPC) vs. the revenue per click (RPC), represented on the *x* and *y* axis respectively:

All very nice, but doesn’t really tell us too much. What we can draw from this is that the relationship is fairly wide-ranging, with some keywords delivering much more in the way of ROI, and some keywords in the bottom-right corner which appear to be unprofitable.

If ever you want to try and add some useful context to a dataset, then segmentation is a really nice, elegant way to achieve this. Instantly (assuming you’ve applied a relevant segmentation), you’ll start being able to compare and assess patterns/trends across different groups, which is often the starting point heading toward the insight that will be useful.

[Read the full article on Search Engine Land.]

The post Visual analysis of AdWords data: a primer appeared first on MarTech.

]]>The post Automate Your Way To Search Marketing Success appeared first on MarTech.

]]>As it’s the time for New Year’s resolutions — high ideals we aim for in order to improve our lives in any number of ways — my advice to you is to resolve to increase the amount of automation used within your SEM campaigns.

Why? Well, my philosophy has always been that an account manager’s strategy will make or break a client campaign. Small changes here and there can nudge performance in the right direction, but an overall vision is what can lead to changes that help performance reach a global, rather than local, maxima.

So it would seem odd to be advocating as much automation as possible. However, it’s a long-held view of many experienced PPCers that getting rid of as much heavy lifting as possible — in this case, by handing it off to an automated process — will free up your time for the all-important strategizing that a machine can’t do.

To help make this point and provide some inspiration for areas where you could lean on automation, here are a few examples (and some resources to help you on your way).

At QueryClick, we use various APIs (AdWords, Google Analytics and more) to populate our reports, allowing account managers to quickly get the data and make sure that the large majority of “reporting time” is put to analyzing the information, digging for insights and planning important actions.

A really accessible way to quickly get the same sort of data — and to create custom, insightful dashboards — is to use the Google Analytics Spreadsheet add-on.

[Read the full article on Search Engine Land.]

The post Automate Your Way To Search Marketing Success appeared first on MarTech.

]]>The post The Search For An Optimal AdWords CPC appeared first on MarTech.

]]>As we are approaching the end of the year, and the time has come to get into a reflective mood, I wanted to return to a nice little problem I’ve mulled over quite a bit this year: What is the optimal AdWords CPC (cost per click)?

Having been in the SEM game for more than 10 years now, you might be surprised that I’m only just thinking about this question — seems quite important, right?

Truth be told, it’s a question I’ve never really stopped thinking about, as I find it very interesting. (It should be noted at this time that my university dissertation focused on an old algebraic parlor game, which was endlessly fascinating but ultimately useless — thankfully, here we’re in more practical territory.)

To help find the optimal solution, we need to have a clear goal. I feel the most obvious goal to aim for is to maximize the profit of a PPC campaign. Maximizing revenue is fine, but its one-dimensional nature can lead you into trouble. (Remember, “Revenue is vanity, profit is sanity.”)

Let me first try and pique your interest in this problem and its shape-shifting answer, then go on to discuss the journey to finding a better understanding of the answer.

A simple answer to a simple question, right?

Let’s look at the most logical way to answer this question, with a simple scenario to get a starting point.

A retailer knows that the conversion rate (CR) on keywords for “Blue Widgets” is 4 percent. He also knows that the average profit per transaction is $90.

Calculating from here means that return on ad spend (ROAS) will be positive for anything below the revenue per click (RPC), which equates to $3.60 (Profit x CR). Let’s look at the dynamic of cost per click (CPC) at different points:

Shock, horror! Increasing the bid leads to negative ROI, whilst decreasing it improves the ROAS. But there we have it — find your break-even point and aim for that as your measured CPC.

But leaving it here would be too easy, and that doesn’t really answer the question at all.

So let’s add a bit more complexity and try and make the situation more realistic and less theoretical. The main issue here is that we are talking about the volume of profit, for which we need to factor in volume of clicks and revenue.

[Read the full article on Search Engine Land.]

The post The Search For An Optimal AdWords CPC appeared first on MarTech.

]]>The post Marketing Analysis: Unlocking The Power Of Descriptive Statistics appeared first on MarTech.

]]>Let me be up front: this post will contain statistics. Not the fun, pithy kind like “*60 percent of statistics are made up on the spot*,” but actual cold, hard statistical practices.

Joking aside, I’m going to run through some fairly high-level statistical analysis practices that you can employ every day to help make sense of your marketing data, allowing better strategic decisions. And it won’t be painful at all, I promise.

*“But I do this already. I’m always analysing my data!”* I hear you say incredulously.

This is probably very true — most users of web analytics (Google Analytics, Omniture, et al.) instinctively apply what would formally be known as “descriptive statistics.”

For example, you readily identify a spike or a drop in your daily traffic by “eyeballing” a chart; you use averages to quickly assess performance; and you do all sorts of comparisons that help you understand what is happening (and importantly, what you need to do next).

Although a loose adherence to the general principles is fine and workable, I strongly believe that an element of rigour can help take your analysis to the next level. Below, I’ll run through a couple of concepts tied to real-world examples that will hopefully convince you that this is an approach you should be considering.

Every set of data has a number of “characteristics” which, when understood, tell you lots about what has happened and what behavior you can expect in the future. One of the major characteristics is the dispersion of data points (i.e., how spread out and different from one another the measurements tend to be).

The formal measure of this is **standard deviation (SD)**, which is derived from its partner metric, **variance (σ ^{2})**. As you’d guess from the names, informally these just represent how much that data can be expected to deviate, and how much it varies. But by utilizing the exact nature of the formal properties, you can do all sorts of

The SD is calculated by taking the square root of the variance. To calculate the variance, we just:

- Work out the average (mean) of your set of results.
- For each measurement you have, subtract the average worked out in step 1, then square the resulting figure. Note each one down.
- Add all the numbers you noted when carrying out step two, and voila! The result is your variance.

To get the SD, just take the square root of your calculated variance. So the whole calculation looks like below:

In finance, SD is a key measure of risk or volatility — it’s incredibly useful to know how stable a portfolio of stocks is before investing. A portfolio may have a high average yield and offer great returns, but if it has a high standard deviation, it could be a risky bet that may make you more averse to committing your money to it.

To think of this within the realm of marketing, consider you are allocating monthly budgets across some campaigns. You have these two, which have potential to spend as much of your budget as you’d like:

With the pure ROI data, it’s simple: spend everything we can on “Snow Shoes,” and put whatever is left towards the “Mammoth Fleece Coats.” (We’ve had the first snow of the year here today, hence the inspired choice of fake campaigns.)

However, if we crunch the numbers and look at the historical deviations, we’ll get some extra context which, depending on our goals, can allow more strategic thinking:

Now, the “Snow Shoes” campaign is still an attractive option; however, the element of risk/volatility needs to be considered. Is a blended strategy (more of a “balanced portfolio” approach) going to ensure more sensible use of your or your client’s budget? It’s entirely dependent on the situation, but this additional data puts you in a much better position to make sound, informed decisions.

Another action upon seeing this data would be to look into the variance. What is the root cause? Are there actions you can take to lower the SD and maintain high performance?

In my introduction, I mentioned that it’s highly likely that you’ve identified upward and downward trends by looking at the time-series charts of your chosen analytics package.

Following through with the formal approach, we can use SD to provide the context around these trends and make really informed decisions by sticking to principles.

In distributed data, we can use the number of standard deviations as a benchmark of how “expected” a certain measurement is — e.g., if we simplify and assume daily transaction data is normally distributed (it probably isn’t, but it’s convenient for me to make this assumption here) then 68 percent of all measurements should fall within one standard deviation, whilst 95 percent should fall within two standard deviations.

What’s the purpose of knowing this? With limited time on our hands as marketers, we need to pick our battles. Using these expectations as yardsticks allows us to categorize spikes and dips as either “business as usual” or “hmm, that’s interesting, I should probably invest time looking into this one.”

Coming full circle and applying this new context to a time series results in what you see below — firstly, a simple standard deviation calculated across the given period, and secondly, a rolling calculation that ensures ongoing updates of the mean and standard deviation.

** **

In our first chart, the end of October shows a consistent dip into the “2 Standard Deviations” zone, so we should be looking into why this is the case. Also, we need to know what happened at the end of August — what can we learn here?

A familiar one to the conversion optimizers out there, **Statistical Significance **is a handy method to inform us if a result we are looking at is caused by some definite relationship, rather than just a quirk of randomness within the data.

For example, when A/B testing and making changes aimed at improving a site conversion rate, you need to be certain* that any measured improvements are attributable to your changes and that the perceived result is not just the data pulling the wool over your eyes.

**Certainty is also a defined measure. You can rarely be 100 percent certain, but you may set your goal to be 95 percent certain that improvements are a result of your awesome site redesign and be happy with that.*

Following are the key components that affect how significant a result is:

**Sample Size:** If you’ve measured something lots and lots of times, you can be more certain about your results. Consider if you have two SEOs and a basketball player in a room, your sample size of three leaves you with very little confidence about the average height of a person. If you measure the height of 1,000 people at random, you’ll be much more certain about how true your estimate of average height is. (It’s not been skewed by the unexplained presence of a Chicago Bull attending your SEO meeting.)

**Standard Deviation: **I hope I’ve succeeded above in conveying what this is. However, the raw SD is not the end of the story. Combining it with the number of measurements (sample size) allows us to calculate the “standard error of the mean (SE).”

*The detail of this calculation isn’t important here and now.* What I want you to take away from this is that many of the results you are looking at can have a degree of certainty (or uncertainty, if you are a pessimist) and that this uncertainty is reliant on how many times you’ve measured something and how varied the results are when you measure them.

This is relevant, for example, when reviewing:

- paid search or programmatic display ad performance.
- the success of an email marketing campaign.
- social engagement rates with different types of content.

The bottom line is that before making decisions based on data, you should understand your level of certainty and how much you can rely on the figures you are looking at.

I hope this has been useful to those who have not encountered these concepts before, and that you’d agree with me that these extra steps and practices are relevant to our work in search marketing.

I’d consider this to be the tip of the iceberg in what I consider to be an exciting, ongoing challenge: making the data tell a story.

The post Marketing Analysis: Unlocking The Power Of Descriptive Statistics appeared first on MarTech.

]]>