Martech: Martech is Marketing Logo
  • Topics
    Digital Transformation
    Marketing Operations
    Data
    Customer & Digital Experience
    Performance Marketing
    Marketing Management
    Special Reports
    MarTech Topics
  • Conference
  • Webinars
  • Intelligence Reports
  • White Papers
  • What is MarTech
    Mission
    Staff
    Newsletter
    Search Engine Land
    Third Door Media

Processing...Please wait.

MarTech » Data » “We Test Everything” And Other Lies Online Marketers Tell

“We Test Everything” And Other Lies Online Marketers Tell

If you don't have a testing regimen in place, you might want to think twice. Columnist David Rodnitzky discusses some common instances where marketers skip testing — and why it's not a good idea.

David Rodnitzky on August 25, 2015 at 11:55 am

ss-split_testing

Online marketers love testing: A/B testing, multivariate, holdouts, incrementality. Heck, before I was married, I’d A/B test my profile descriptions on dating sites. (For some reason, my best results came when I posted the blurriest photos … kidding.)

The beauty of online marketing is that we get reams of data, almost instantly, that enable us to quickly test iterations to improve our results. Nerdy as it may sound, it is exciting to launch an experiment and sit in front of the computer and watch the results show up on a dashboard, right? Who’s with me?

I firmly believe that the best online marketers are the folks who are religious about testing every decision possible. More often than not, however, I’ve observed marketers who only selectively implement a testing regimen.

This article describes some of the most common instances where marketers eschew testing — and why their reasons for doing so aren’t good enough.

Testing Fail #1: We’re At Maximum Efficiency

Like any team inside a company, marketers have to fight for budget, be it annually or quarterly. Once budget is established, the marketing team is often faced with the perverse “use it or lose it” choice, where reducing budget (even if for smart reasons) results in the marketing team getting “punished” with less budget in the next cycle.

In a true testing environment, use it or lose it should never be a factor in deciding the optimal budget. Marketing teams should communicate to senior leadership that part of their responsibility to the company is to test different budget amounts to determine the optimal efficiency and that budget for the next year shouldn’t be reduced just because not all of the prior year’s budget was used.

At One Kings Lane (a former client), for example, VP of Marketing Jim Kingsbury routinely schedules an “efficiency month” where budgets are slashed from every channel and each channel team is directed to drive the same amount of revenue from significantly less budget. This exercise forces teams to think creatively about channels and tactics and drives tests that often improve efficiency.

Testing Fail #2: Our Agency/Team Can’t Be Beat

Humans are strongly affected by “confirmation bias,” or according to the definition on Wikipedia, “the tendency to search for, interpret, favor, and recall information in a way that confirms one’s beliefs or hypotheses.” This is most apparent in marketing organizations when it comes to choices about staffing.

I’ve seen this manifest itself in two ways:

  • Switching budget from an agency to an in-house team without testing
  • Replacing an agency without any form of bake-off

For example, let’s say that a new VP of marketing joins a company with an existing agency relationship in place. The VP then hires some in-house team members who subsequently tell him that they can manage the marketing better than the agency.

Since the VP hired these team members (but inherited the agency), the VP subconsciously concludes that his team must be better than the agency, because the internal team is a reflection of him and the agency is not. The same situation occurs when the VP replaces the agency with an agency he has worked with in the past.

Marketing experts who truly believe that testing drives the best results eschew confirmation bias, or at least try to dampen its influence. So in the case where the in-house team claims that they can outperform the agency, a test-driven VP would give half the budget to the agency and half to the in-house team to see which one truly was the best.

For more on the general decision of in-house versus agency, see my last Marketing Land post.

Testing Fail #3: We Don’t Have Time/Budget For New Channels/Technology

Marketing teams are under constant pressure to deliver better and better results. This often leads marketers to focus all of their attention on “what’s working now” with little regard to “what could work better in the future.”

I see this most frequently when it comes to new channels or technology. From a channel perspective, I believe the job of a marketer is to figure out where customers are online and then market to those customers.

Customer behavior online changes very rapidly, and that generally means that a marketer’s media mix should evolve with equal speed. And yet, marketing spend almost always lags behind customer behavior.

If you don’t believe me, just look at the amount of money that is still spent on print and TV versus the amount spent on mobile advertising, and then compare that to actual consumer usage.

The same hesitancy to try new channels applies to technology. Having to talk to the IT team about implementing a new pixel (much less an entire new software system), juxtaposed with comfy familiarity with an existing system, leads marketers to avoid new technology vendors like the plague. (In economics, this is referred to as “switching costs.”)

A marketer committed to testing would set aside both time and budget to new channels and technologies, always trying to find the next great thing that will move their business forward.

Don’t Be A HIPPO, Be A Tester

I once quipped that the biggest challenge facing online marketers today is “mo’ data, mo’ problems.” There is not enough time in the day to look at all the different data sources available to us.

As most marketers have discovered, the solution to this overabundance of information is to be selective about what you choose to analyze. There’s a huge difference, however, between selecting testing and no testing at all.

A complete lack of testing leads to decisions by HIPPO, “the highest paid person’s opinion.” And a hippo who is also influenced by confirmation bias, budget pressure and switching costs is unlikely to make the most efficient decision for a business.

My friend Chris Goward from Wider Funnel wrote a book recently that has perhaps the most perfect title of all time: “You Should Test That.” At the end of the day, marketers who truly believe in the power of testing should live by this adage when asked for their opinion on any topic.

Should we replace our agency with an in-house team? You should test that. Should we change ad servers? You should test that. Do I need more budget? You should test that!

Don’t be a hippo, be a tester!


Opinions expressed in this article are those of the guest author and not necessarily MarTech. Staff authors are listed here.


New on MarTech

    Processes, visibility and optimization: The keys to digital marketing transformation
    Webinar: Grow revenue streams through web accessibility and compliance
    How to turn the great buyer resignation into B2B career opportunities
    HubSpot customers paying 12% more than in 2021
    Closing your team’s technical gap without hiring

About The Author

David Rodnitzky
David Rodnitzky is CEO and co-founder of 3Q Digital, a marketing firm with offices in the San Francisco Bay Area and downtown Chicago. David is the founder of the LinkedIn Online Lead Generation Group, an advisor for Marin Software, and a regular contributor to the 3Q Digital blog. He can be found at numerous speaking engagements across the SEM community.

Related Topics

DataDigital Transformation

Get the daily newsletter digital marketers rely on.

Processing...Please wait.

See terms.

ATTEND OUR EVENTS The MarTech Conference logo.

September 28-29, 2022: Fall

Start Training Now: Master Classes

Start Discovering Now: Spring



The SMX Conference logo.

Start Training Now:: SMX Advanced

November 14-15, 2022: SMX Next

March 8-9, 2022: Master Classes

Webinars

Agencies: Grow Revenue Streams Through Web Accessibility & Compliance

Protect Your Paid Advertising Spend Against Ad Fraud and Invalid Traffic

Build an Integrated Search Strategy Across Google, Amazon and YouTube

See More Webinars
Intelligence Reports

Enterprise SEO Platforms: A Marketer’s Guide

Enterprise Identity Resolution Platforms

Email Marketing Platforms: A Marketer’s Guide

See More Intelligence Reports
Featured White Paper

Digital Marketing Strategy Ebook

See More Whitepapers

Receive daily marketing news & analysis.

Processing...Please wait.

Topics

  • Transformation
  • Operations
  • Data
  • Experience
  • Performance
  • Management
  • All Topics
  • Home

Our Events

  • MarTech
  • Search Marketing Expo - SMX

About

  • What is MarTech
  • Contact
  • Privacy
  • Terms Of Use
  • Marketing Opportunities
  • Staff

Follow Us

  • Facebook
  • Twitter
  • LinkedIn
  • Newsletters
  • RSS

© 2022 Third Door Media, Inc. All rights reserved.