Data-Driven Marketing: A Test-and-Learn Culture

Tom Affinito, VP Corporate Development @ Kenshoo

October 09, 2019

A test-and-learn culture is a key requirement for data-driven marketing. After all, being “data-driven” can’t just be lip service—it has to be fundamental and core to your entire marketing organization and the only way to truly foster and facilitate a data-driven approach is when the team is committed to the process, the right investments in technology and processes are in place, and everyone is aligned and accountable.

Data-driven marketing has become the aspirational principle for organizations to increase performance while reducing the risk from basing decisions on ad hoc processes fueled by opinions and guesswork.

But the continued increase in global media spending (at $665 billion this year and growing to $855 billion as soon as 2023) and the further need to align this media impact with business outcomes is putting more scrutiny on justifications for these media plans, especially in the disruptive and dynamic markets in automotive, financial services, retail, ecommerce, CPG and travel verticals.

Marketing experiments point marketers in the right direction

In the recent Kenshoo white paper Marketing Experiments: How Leading Companies Make Data-Driven Decisions, marketing experiments are seen as a key approach to improve data-driven execution by enabling marketing leaders to:

Validate and calibrate their data-driven decision making
Identify the right data sets required for a given decision
Test transformative ideas for viability and gain valuable insight before fully committing

Marketers are starting to realize just how important testing is to making great decisions. According to Gartner’s 2018 Marketing Data and Analytics Survey, 67% of marketers say experiment design and test design will become more important within the next five years and Econsultancy reports than nine in ten (91%) company respondents expect to increase the number of experiments they run over the coming 12 months.

Today’s largest media companies, including Amazon and Google, have mature test-and-learn processes ingrained into their marketing decision-making…but the hard truth is that many brand marketing teams are simply not trained to hypothesize, discover and extract insights from data. Marketing experiments sound good on paper, but outside of short-term publisher-specific A/B tests, most brands struggle with even the basics.

How can we increase our own team’s capacity for executing marketing experiments, and drive further success in operational planning, discovering strategic insights, and aligning with business outcomes?

Download this complimentary white paper to learn more

Test-and-learn culture: supporting marketing experiments and data-driven maturity

The best way forward for marketing leaders to be more data-driven is to focus on building a test-and-learn culture. This approach cultivates a practice in which team members are armed with the right skills, incentives, processes, and tools to incorporate marketing experiments and to be truly data-driven in actions and not just words.

A test-and-learn culture is one in which the entire organization:

Understands the importance of data-driven decision making, with each team member empowered to play a role in more deeply measuring and monitoring media impacts and influences
Prioritizes experiments, recognizing when risky less-predictable environments or audiences require the development of new data insights
Ensures that everyone on the team is trained in data handling, hypothesis building, insight building, and marketing experiment methodologies
Commits to process-building and technology investments to foster best-in-class data intelligence
Understands that experiments will often fail and that only proven insights will reliably drive better business actions

In a test-and-learn culture, most of the challenges to successful data-driven marketing can be overcome or reduced. And because data-driven decision making is inherently more rigorous than ad hoc or purely-historical decision making, a test-and-learn culture helps to foster a level of alignment and understanding throughout the business because everyone is on board and accountable to the insights, metrics, and key indicators delivered.

Making the move to a test-and-learn culture is worth it

Migrating to a test-and-learn culture isn’t an easy shift, but according to Gartner, high-performance companies that make testing and experimentation a marketing priority reap three significant advantages:

“The reduction of abundant opinion and guesswork from marketing execution. Supported by documented customer insights, approaches become clear for improving effectiveness across advertising, web and mobile initiatives, for example.
The ability to react quickly to changes in customer expectations, including any competitors’ enhancements. Agile testing methodologies accelerate your ability to assess the value of competitors’ enhancements and minimize your market vulnerability.
The creation of disruptive threats and new revenue streams. Teams that test all hypotheses — whether conservative or outrageous — increase their likelihood of revealing a hidden revenue gem.”

Test-and-learn culture: examples from marketing leaders

Expedia: no guesses, just tests

From the CMO.com article, Five lessons from Expedia’s test-and-learn culture

“For online travel company, Expedia, there is no good or bad idea, you just need to test them. At any one point in time, brand Expedia is working on 50 different versions of the website and providing daily updates to its customers. Its engineering teams can detect small changes and figure out what version a consumer really likes using data insights, then adapt quickly to suit.

‘We don’t have to guess, just test,’ said Expedia’s former CEO in a 2015 interview. ‘What we find is one-third of ideas work, one-third are things the consumer doesn’t care about, and one-third fail. The difference between failure and success can be so small. You need to harness audiences that over time will show if you it’s a good or bad feature.’”

Clorox: tests for what’s needed and what’s possible

From the Gartner article, Clorox’s Test and Learn Approach to Building a Martech Roadmap

“Marketers often fail to connect prioritization and planning efforts with the practical realities and challenges of technology. Moreover, emerging technologies with the potential to reshape the consumer journey and experience tend to be overlooked.

To tackle these challenges, Clorox has two kinds of in-depth learning projects — ‘what’s needed’ learning projects and ‘what’s possible’ learning projects.

‘What’s needed’ projects consist of mapping the consumer journey, investigating areas where marketing capabilities do not meet the consumer need and developing and testing solutions.

‘What’s possible’ projects involve investigating where to apply emerging marketing capabilities to create valuable new consumer experiences and developing and testing prototypes.”

Accenture: widening impacts through cross-functional test-and-learn

From the Accenture article, Experimentation and Unlocking Data-Driven Innovation

“Use each innovation hypothesis to direct your efforts to create a targeted experiment. That can then be tested, via a series of real-world scenarios, with data from each test being used to drive decisions.

A common pitfall with these types of experiments is a narrow focus on just one element of product or process. For example, developers may be pushing to make their app faster, but performance speeds may have no positive impact on the business’ most pressing problems. To truly reflect the business, hypotheses should be developed across cross-functional “pods” that focus on specific problems but have input from different entities spanning the organization.

The real value comes with a team at the intersection of business, product, and operations. For a company working on a video product, this means bringing together product management, marketing, customer service, and development. When they create a hypothesis, they’ll have a holistic view of the problem, avoiding a narrow focus.”

Test-and-learn culture: the pillars of success

In conversations with our clients, four key pillars come up time after time in discussing how to improve test-and-learn culture: a) creating robust testing processes, b) bolstering the dissemination, management, and monitoring of insights and their derived policies, c) supporting teams to recognize and prioritize strategic initiatives, and d) aligning the focus and domain of marketing experiments with the vision of business.

Join us next week as we explore these four marketing test-and-learn foundations and see where your own team can focus to achieve even more data-driven success!