Lior Feldman, Product Manager @ Kenshoo
Being data-driven in decision-making is the expectation for most marketing teams today. Yet it only goes so far when the data referenced is historical in nature and limited in source to campaign and performance data.
The next level of data-driven decision making is to test & learn, a practice that can empower marketers to leverage what they know in order to make informed and educated guesses in order to discern what they don’t—or even can’t.
By building assumptions and running tests to see if they are validated, marketers can glean tremendous value, for even learning that an assumption was wrong becomes both meaningful and actionable.
At Kenshoo, we believe that a key tenet about the future of marketing is that all members of the marketing team will be experts in the test & learn approach with marketing experiments at the foundation of that practice. The media team will always be testing bidding, the creative team will always be testing messaging, the targeting team will always be testing targeting, and even the CMO will run tests to guide channel budget allocation, media mix and so on.
For more information, read the Kenshoo report: Marketing Experiments: How Leading Companies Make Data-Driven Decisions.
Search marketers have had access to publisher testing solutions for some time now with Google’s Campaign Drafts and Experiments and Microsoft Advertising Experiments. Using these tools, practitioners can execute standard A/B tests to experiment with campaign iterations to determine what variations work the best.
Those tools are a step in the right direction, but Kenshoo Experiments offers search marketers using Kenshoo Search a set of advanced features to enhance experiment setup, management, and reporting. By utilizing these features, running experiments as a core approach helps marketing become leveled up and more valuable to the organization.
Multi-campaign support. With Google’s Drafts and Experiments, search marketers can only test one campaign at a time in the A (control) and B (test) groups. With Kenshoo Experiments, marketers can add multiple campaigns to each of their Google experiments.
One of the main advantages of multi-campaigns is that one campaign may not have enough volume to reach statistically significant results. By grouping campaigns running the same test, it helps to get to conclusive results much faster.
Precise control over test parameters. With the publisher tools, a marketer sets start and end dates and the experiment runs. With Kenshoo Experiments, the practitioner can set the baseline, ramp-up, and test periods.
Allowing the time needed to set a clean baseline is a foundational best practice to marketing experiments so that you have the pre-test numbers to compare with the test results to ensure that nothing out of the ordinary happened during the test that might skew results. With the publisher tools, marketers have to export the pre- and post- data and manually aggregate it in a spreadsheet themselves.
Choose your tests’ baseline, ramp-up testing timelines
Testing custom metrics. Every marketing organization is different and the standard KPIs hardly ever capture what those teams care most about. Often, there are custom metrics which our clients are most focused on and Kenshoo Search is able to integrate our client’s backend data so that they are able to report and optimize on directly within their advertising campaign management tool instead of having to pick a publisher “off the rack” metric that is usually just a proxy to true performance.
Kenshoo Experiments allows Kenshoo clients to run marketing experiments on these custom metrics.
Rich, visual reporting. To date, publisher experimentation tools offer fairly simple views of results. One of the parts of Kenshoo Experiments that most clients often love when they first see them are the various charts and graphical views with day-over-day performance metrics.
Kenshoo Experiments offers at-a-glance insights
Data exclusion. As anyone who has run many marketing experiments knows, there are often anomalies that pop up that can skew test results. For example, if a tracking code was mistakenly deleted, a landing page URL accidentally switched, or just something happened in the world that spiked or dropped conversion activity for a few days.
In most of these cases, the marketing experiment has to be stopped and started over. With Kenshoo Experiments, the practitioner needs only to identify those dates for the system and that data will be excluded from the analysis.
Easily exclude data from dates you choose
Advanced device reporting. Kenshoo Experiments shows you results for your experiments across devices. Why shut a campaign off if it looks like it has average performance, but in reality it’s not working well on one device type while working really well on another? Aggregated/blended results might mean a practitioner might pause the campaign whereas a device split report might have them pause one device type so it can benefit from a device-oriented strategy.
As more and more marketing organizations begin to rely on a test & learn approach with marketing experiments foundation at its core, the need for more features to enhance Kenshoo Experiments are requested.
As the product manager on Kenshoo Experiments, I’m excited about some of the next feature sets that we have planned on the roadmap including some near-term things which are set to release this summer.
Some of those include:
Would you like to see Kenshoo Experiments in action?
Contact us to see a brief demo of the tools mentioned in this article.
Request a Demo of Kenshoo