Jason Puckett

July 1, 2014

What is Multivariate Ad Testing?

Digital advertising is an ever-evolving environment, competing for ad space and a user’s attention is becoming more and more difficult. Evoking a desired user response through paid ads is an art form and only data driven decision makers survive. Continuous improvement to your display, search & mobile ads has never been more important for driving revenue, downloads, page visits and/or lead generation. Regardless of your desired outcome, the top of your funnel depends on reaching the optimal quality of your ads.

Multivariate testing & analysis is a methodology that enables advertisers to continuously increase ROI and gain deep insights about customers, products, markets and messaging. Continuous experimentation and improvement is the core of multivariate testing and is absolutely essential to the process of maximizing ROI.

Search, display & mobile ads are comprised of several key variables: keyword group, message, call-to-action, extension, targeting, page positioning, etc. Each of these variables (or elements) interacts with one another. Some of these ad combinations will drive conversions and clicks while others will not.

Multivariate testing allows marketers to determine the optimal combination of these elements by running live experiments with real users. After the first few tests, marketers can zero in on the most important variables, which streamlines the testing process and results in better ads, faster.

To run a multivariate test, advertisers need to define the relevant variables, iteratively test various combinations in a live environment, and measure the results. Testing all possible combinations is known as full factorial testing. It may seem somewhat overwhelming at first, but with AdBasis this process can be managed easily and yield huge returns.

Lets Look at an Example

Company A sells clothing through an online store. It is targeting customers across 10 US states and the marketing team is creating a paid search campaign to drive sales. The marketing team believes regional messaging will be effective but is unsure how to best customize its ads for its target markets.

Company A would like to test the following variables within their search ads:

  • 5 titles
  • 5 messages
  • 5 calls to action
  • 4 product landing pages
  • 10 states

That is a total of 5,000 potential ad combinations (5x5x5x4x10) that Company A would need to create and test in order to find the optimal mix of variables. Granted, this would be a large experiment but it would yield valuable insight and provide a drastic improvement to ROI for the advertiser.

Within AdBasis, Company A would create all 5,000 ad permutations in minutes, instantly launch the experiment, manage budget and the marketing team would start seeing live results. As soon as each ad has been exposed to a statistically significant number of real people, AdBasis selects the “winning” ad and Company A’s ad budget is automatically reallocated to the optimal advertisement. Now Company A’s job is to run another experiment to beat the winning ad for their campaign.

Potential Uses of Multivariate Testing

Multivariate testing is most commonly used for advertising in instances when multiple variables within the ad need to be tested (which region is targeted, message within the ad, how it appears, etc.). The number of variables tested usually depends on budget, time and desired results. AdBasis removes much of the guesswork by providing pre-determined estimates of budget, time required, and confidence level for each experiment before launch.

The Goal

The goal of multivariate testing is to empower marketers with the rich data they need to make better decisions! After a multivariate test concludes, a clear depiction of the results and an understanding of the relationship between the different elements within your ads emerge. Multivariate tests gather huge amounts of data and the analysis of this data conducted within AdBasis can provide deeper knowledge into your customers’ online behavior.

After you have created the “winning” ad, it’s time to leverage this understanding into improved experimentation. Continuous testing and constant improvement is vital to helping your company’s bottom line over time.

AdBasis is the leading technology provider that makes multivariate testing easy & automated. The potential increase to your CTR, ROI & conversion rates is waiting, it’s all up to you to start testing!

Have a question for us? We'd love to hear from you!

comments powered by Disqus


Popular Articles

Understanding Conversion Layers and Executing Ad Experiments

August, 14th 2014 - Optimization

3 Steps for Building a Paid Acquisition A/B Testing Plan

October, 15th 2015 - Optimization

All Things Tested - 3Q Digital's Sean McEntee

August, 31st 2015 - "All Things Tested"