Jason Puckett

May 25, 2016

AdWords Search Ads A/B Testing Guide

A/B Testing AdWords Search ads is the most effective way to successful campaigns. In this guide you'll learn how to effectively test your advertising for AdWords.

Guide 1

What is an A/B Test for AdWords Search Ads?

A/B testing ad copy for AdWords search ads is a unique way to apply A/B testing theories. Traditionally, most marketers focus solely on A/B and multivariate testing for landing pages. However, testing AdWords search copy can provide an equal if not greater return and must be a focus of digital marketers.

If AdWords search ads are a big part of your marketing mix, executing these tests is an absolute must. As you may already know, A/B ad testing is when you alter a single element within an ad, and measure the change of that element. Multivariate ad testing is used when multiple elements are up for debate.

Let’s look at an example, this shows us the sample size (don’t be scared, this is one of our larger customers), the differences in the ad copy, and the increase in performance metrics.

Guide 2

Here is what the test structure looks like within each ad group; all possible permutations (4). 2x description line 1’s and 2x landing pages.

Guide 3

This same test takes place across 400 ad groups at the same time (the above structure is inserted into 400 ad groups at once), the data is aggregated and performance calculations are shown for the entire ad group set. The total test structure looks like this:

Guide 4

The goal is to then aggregate the data for each permutation across all ad groups and measure it, and calculate the statistical significance of these findings. As shown here:

Guide 5

Then, break down the rest results for each variable individually:

Guide 6

Marketers who have previously performed A/B tests on their search ads within an AdWords account understand the difficulties of managing that process. Performing and interpreting regression analysis of variable results is next to impossible. Without a set process for your experiment, you cannot expect consistent ROI improvement.

You can either: (1) continue to do this manually or (2) use an ad testing platform like AdBasis to improve your ROI and save time spent ad testing by 90%. Yes, shameless plug for AdBasis.

What is Testable in AdWords Search Ads? The 3 Types of Variables

One major misconception is that ad testing is related solely to ad copy, and that landing page testing happens in isolation from ad creative testing. At AdBasis, we focus on testing both of these together, which we refer to as “Full Funnel Testing”.

There are three types of variables that can be included in your ad tests:

  1. Ad Copy - Headline, Display URL, Description 1, Description 2, Ad Extensions.
  2. Your Targeting - Make changes to your demographics, ad groups, devices, geos, etc.
  3. Your Final URL (Landing Page) - Testing alternate landing pages for specific ad group sets is a key component to the truest form of ad testing.

Ad Copy Testing vs Traditional Landing Page Tests (hint these should happen together):

  • Easier to implement than traditional landing page tests
  • Obtain results more quickly because the sample size is impressions rather than web visits
  • More versatile than traditional landing page tests because this can be for a specific subset of keywords, ad groups or campaigns.

Guide 7

How to Test Google Search Ads In AdWords

We have outlined our proven 7-step process for ad testing:

Guide 8

The Framework for Testing & Optimization

Testing and optimization is a continuous process. It is best to think of ad testing as a cyclical experiment that involves gathering quantitative data to support qualitative theories you may have.

Guide 9

1. Identify Your Primary Opportunities

Once you’ve identified the KPI’s you’re interested in improving, you must look for areas to make these improvements. Where can you find these opportunities?

Your AdWords Account

An analysis of your existing AdWords account (audit) can provide some really good areas of opportunity to begin testing. Things to consider:

  • Top performing sets of ad groups
  • 5 ad groups with largest discrepancy between CTR and conversion rate
  • 5 ad groups with the highest CPC
  • 5 ad groups with the lowest ROI
  • Landing page with the lowest overall conversion rate
  • 5 ads that are driving poorly converting traffic to your lowest performing landing page

These examples are all great quantitative starting points for learning. Use your knowledge of your marketing calendar to then prioritize the quantitative needs.

Your Marketing Calendar

Your marketing calendar should tell you what you need to be testing. If you have a new product being released, test the advertisements for this promotion prior to the launch. If your business has a busy season that begins at the end of November, start ad testing one month beforehand. Understand which KPIs are most important before the demand spike and hit the ground optimal.

Guide 10

2. Structure Your Ad Testing Plan Based on Desired Outcomes

Select what you would like to learn, and work backwards. Remember, the primary purpose of ad testing is to find conclusions upon which you can rely. Begin your experiments by thinking "I’d like to know __ ".

  • “I’d like to know which creative sells product X to users in the US”
  • “I’d like to know which of these two landing pages works best for mobile, ad group set X”
  • “I’d like to know which of these Description 2 calls-to-action is best for converting my mobile traffic.“

3. Prioritize and Create Experiment Plan Doc

Which pieces of information are most crucial to you and your department? What are your ‘bread and butter’ products? How are you and your team measuring success? The timing and prioritization of your experiment calendar should reflect what is important to your brand.

Develop a calendar specifically for ad testing. Use your previous data to project how long ad tests will take and build a GANTT chart using a calendar to remind yourself when to end or launch your experiments.

4. Monitor Ongoing Statistical Significance Calculations

Calculate your statistical significance levels at least once every-other day. If possible, use a platform like AdBasis to keep an ongoing calculation. This will allow you to continuous re-allocate spend to top performing variations, which is the ultimate goal.

AdBasis allows you to delete underperforming ad variations mid-experiment. Be aware that deleting ads too early can harm the validity of your experiment. Use this ability in context of your ad budget.

If the budget is tight, consider deleting the bottom 25 percent of variations in the experiment once the top 25 percent have reached a 70 percent confidence level. Let the ads performing in the top 75 percent of the original experiment battle it out.

5. Uncover Winning Variations

After each variation within your experiment has been exposed to a statistically significant number of users, the winner will be determined. Keep in mind that some experiments will not yield a winner.

6. Regression Analysis - Understand Why

Regression is impact which one variable has on specific KPIs. The goal of segmenting within using regression is to be able to unequivocally say: "this variable makes the biggest difference, and here is the optimal variable combination for my conversion goal".

Here’s a Tremendously Valuable Example of Regression:

Company X ran an experiment targeting users in California; they have two California-specific landing pages. They created 20 versions of an ad and served them equally to each landing page across two schedule groups (Evening_Weekday, Morning_Weekday).

The Goal: Find the time of day for weekdays that works best, and establish which landing page works best.

When looking at the two segments below, Weekday_Morning vs. Weekday_Evening, one of the schedule groups dominates.

Guide 11

Weekday Mornings is the best performing schedule group for every metric, right? Let’s take a deeper look.

We have added another segment to the graph and uncovered some valuable information. We have now added performance for each landing page and compared these to the two schedule groups.

Guide 12

We now see that most of the success from “Weekday Mornings” is attributed to Landing Page 2 and “Landing Page 1- Weekday Mornings” actually performs worse than “Weekday Evenings” as a whole.

If we merely looked at the first graph, we assume that Weekday Mornings are better than Weekday Evenings. However, when we add some granularity, we see that the success of Weekday Mornings is dependent on Landing Page 2.

If Company A had not added a second landing page to the experiment, they would have received completely different insights and may have believed that Weekday Evenings perform best.

The point is simple: a deeper analysis of individual variables is necessary in order to make advertising decisions.

This data can be leveraged for Company A’s next experiment.

7. Enhance Testing and Beat The Winner

As you know, ad testing is an iterative process. You know that versions should be based on data. AdBasis has created a scientific process for creating experiments and gathering data. Our 7 step process has hopefully given you some insight on how to approach your ad testing strategy. Take a look at the data you’ve gathered, improve your test and build another.

Keep testing!

Have a question for us? We'd love to hear from you!

comments powered by Disqus


Popular Articles

Understanding Conversion Layers and Executing Ad Experiments

August, 14th 2014 - Optimization

3 Steps for Building a Paid Acquisition A/B Testing Plan

October, 15th 2015 - Optimization

All Things Tested - 3Q Digital's Sean McEntee

August, 31st 2015 - "All Things Tested"