Advanced testing: What 4,000 A/B tests can teach you
Contributor Jason Puckett summarizes a presentation from SMX Advanced 2017 on lessons learned from 11 years of conversion optimization. In it, speaker Ayat Shukairy explains what's critical for the success of any A/B testing program.
SMX Advanced has always featured the industry’s top talent discussing what they’re passionate about: search marketing. Last week’s conference in Seattle was no different.
Ayat Shukairy, co-founder and vice president of client solutions at Invesp, shared nine lessons from 11 years of A/B testing experience. The goal of the presentation was to present the many nuances which go into a successful A/B testing plan. After 4,000 successful conversion rate optimization (CRO) implementations, Shukairy has developed quite a knowledge base.
Below I have listed Shukairy’s nine key takeaways, along with some additional insights and commentary:
1. We often focus on the wrong thing
Shukairy advised us to stop focusing on the short term. We need to keep our long-term testing plans in mind when executing our strategies.
Additionally, Shukairy reminded the audience to be skeptical of giant lifts in performance; if you see a huge lift in conversion rates (CVR), then check to make sure that (a) you’ve run the test long enough to produce meaningful results, and (b) this test is not an outlier.