Jason Puckett

February 10, 2017

We A/B Tested $45M in Google Search Ads - Here's What We Learned About Headlines

A/B testing AdWords search ads is hard… but this study will make it just a bit easier. Through improvements in advertising technology, uncovering optimal ad creative for every audience and “scenario” is not only possible, it’s a must.

Since Google released the Expanded Text Ad (ETA) back in August, we have all wondered how to write the best possible ads in order to maximize the newly available ad space. If you’re asking the question “What Works Best in ETA Headlines?” this article is the most helpful piece of content you’re going to read on the topic. Yes, lots of data. We are all going to need to make this transition by January 31st if you have not already done so, so get ready for some knowledge.

…no this study won’t tell you what’s going to improve your conversion rate, you will need to test it for your own brand. What we can do is provide a very helpful creative guideline and the data to back it up. Ad copy testing is one of the most underutilized ways to optimize your paid acquisition funnels, this is a guide to fuel your testing plan.

Why Are We Looking at Headlines?

Because users look at this first! Arguably the most important ad copy variable in existence. It’s the biggest, it takes up the most digital real estate, it may be the first thing your potential customer will read. How can you make sure you’re maximizing your efficiency? You’ve probably tested some stuff but is it really optimized? Let this article help you.

Let’s pretend you’re an advertiser, let’s pretend you’re buying search ads. How are you going to leverage your headline to entice someone to click on your ad? The AdBasis database contains the creative metadata from over 7.6M search ads… we stored this data, we tracked it, and can now show you what’s worked. Let’s look and see what happened (data is anonymous to protect the innocent).

The Sample Size of our Study

Before you read the below figures and you start trying to calculate the number of impressions per test or per ad, keep something in mind. Our customers have very large AdWords accounts. Typically a single ad test will be distributed to several hundred or even several thousand ad groups. Therefore, a single creative variation may count as several hundred or several thousand ad UNITS. We can explain how we do this without sacrificing ad relevance but that’s a topic for a different time.

#FunFact - It would cost you $45.1M to get this much creative data, save your money, just keep reading.

Expanded Text Ad Units (Search) - 7,613,605

Search Ad Impressions - 21,467,181,546

Clicks - 326,653,266

Conversions - 13,032,414

Number of Ad Tests - 5,965

Dollars Spent - $45,118,212

Deciding on a Headline Structure

Before you enter the creative process, map out the purpose of each headline. Your ad copy variations will need to test different ad copy manipulations but also different creative approaches. Certain topics will work well for certain areas within your ads, test them!

A strategic purpose for each headline is a very important part of the process.


  • Company name (used when you have high brand recognition or under branded terms)
  • Product concept statement (stating what you do or what you are)
  • Differentiator (how your product differs from others)
  • Call to Action (what you want the user to do)
  • Value Proposition (what your product solves in terms of benefit to the clicker)
  • Promotional Language (Pricing, Discounts, Inventory Numbers, Free Offers, etc.)

Headline Test 1

How to Start a Headline 1 or Headline 2

You’re a copywriter, you sit down to write some ads. How do you begin? You begin by writing a word. But what words should your Headline 1 (H1) or Headline 2 (H2) start with?

Well, we looked at some of the most popular “first words” in the 7.6M ads we have in our database… we included the sample size and the associated performance of ads where the H1 or H2 start with said word.

Sorted by CTR but also includes Conversion Rates and Sample Size…

Headline Test 2

Graph Sorted by CTR

Headline Test 3

What Should My Headline 1 Contain?

Once you’ve decided how to begin your H1’s and H2’s, it’s time to write. You need to decide the actual topic of your Headline 1. Defining a strong purpose for both your H1’s and your H2’s is a very important step prior to writing them. We outlined this step above.

As you work through your creative purpose for each variable - here is what other brands are doing and how effective it is at obtaining clicks and conversions. You can also see the volumes related to each creative concept.

Headline Test 4

Sorted by CTR if Headline 1 “Contains”

Headline Test 5

Sorted by Conversion Rate if Headline 1 “Contains”

Headline Test 6

As you can see, as far as generating clicks and grabbing attention, using the words “Official” and “Order” work extremely well.

#FunFact - Using Dynamic Keyword Insertion in your Headline 1’s does not necessarily improve your ad relevance, positioning or CTR.

No matter what you decide the purpose(s) of your H1’s should be, please use this guide as a creative fuel!

How to Write Your Headline 2

You’ve written your Headline 1, nice work! Now use that newly added digital real estate (H2) to really send a message.

The main point here is that the purpose of each Headline should differ. Don’t use more space saying things that are somewhat repetitive (see what I did there?). The order and the ideal topics are somewhat difficult to pinpoint, be sure to test different combinations.

Here’s what other companies are doing…

Headline Test 7

CTR if Headline 2 “Contains”

Headline Test 8

CVR if Headline 2 “Contains”

Headline Test 9

You can see here that the same words and phrases used in H1 have different performance levels in H2. What does this tell us? That these two areas of your ad have different purposes, there is something about how users see the ads that drive them to do different things.

#FunFact - The same creative phrases perform differently in H1 and H2. Be sure to pinpoint which is optimal.

Data for Your Own Ads

As we have stated before, this study should not be used as gospel, of course as we drill down by product category, demographic, geographic region etc the creative performance of these elements will change. This won’t necessarily be applicable to your own brand, you will need to test, but this should hopefully fuel your creative fire.

As always, testing is a very important component to any marketing channel. Creative testing for search ads has become much more achievable and is quickly becoming a requirement in order to stay competitive. SEMs everywhere can hopefully take some of this data and apply it to their own testing plans.

Have a question for us? We'd love to hear from you!

comments powered by Disqus


Popular Articles

Understanding Conversion Layers and Executing Ad Experiments

August, 14th 2014 - Optimization

3 Steps for Building a Paid Acquisition A/B Testing Plan

October, 15th 2015 - Optimization

All Things Tested - 3Q Digital's Sean McEntee

August, 31st 2015 - "All Things Tested"