How to keep display ad A/B testing from blowing up in your face

This guest post is by Myles Younger, co-founder of Canned Banners. Canned Banners provides a platform and tools for automating and streamlining display ad design. Follow the company on Twitter at @cannedbanners.

Online advertisers, especially search marketers, understand the power of A/B testing. In search, the most successful advertisers will constantly A/B test hundreds of different ads. With the recent growth in display advertising, it’s logical for online marketers to try and apply their A/B testing expertise to the world of display.

However, data-driven marketers should be careful before diving into display advertising. This post is going to explain some key differences between search ads and display ads, and offer some tips on how to keep your display ad design budget (and your sanity) under control when you’re doing large-scale A/B tests.

Notes: I’m not including Facebook ads when I say “display ads.” That could be a whole other blog post, and I’m not really qualified to write it. Nor am I considering landing page optimization, which is a critical factor in ad testing, but I needed to keep my focus limited.

Search ads are simple. Display ads are complex.

Before we go further, it’s important to appreciate just how much more complex a display ad is versus a search ad. Here’s some rough math to give you an idea of the difference in complexity:

Why is complexity important? With search ads, your creative palette is constrained to one element: text. With display ads, you could potentially test an infinite variety of elements: text, color, imagery, fonts, animation, button style, etc. So while it’s almost impossible to create an “ugly” search ad, it’s very, very easy to make poor choices and create an ugly display ad.

Tip #1 — Start with good designs

Even though you might need to test potentially hundreds of different ad variations, don’t cut corners and launch with garbage ad creative. Finding out that Crappy Display Ad A beat Crappy Display Ad B is like learning that horse manure smells slightly better than dog poop. Don’t waste your money & time.

If you think you can randomly test crappy designs and eventually iterate your way to a perfect, beautiful display ad, read the previous graphic again (and maybe study up on how exponents work); the universe won’t be around long enough to perform all the necessary design iterations to hit paydirt. Launch with thoughtful, professional ad designs that you feel good about.

This doesn’t mean you need to spend weeks & weeks designing your first round of ads, but don’t just throw some clipart and tacky text effects into a box and call it a display ad. In general, ugly ads won’t perform very well and they’ll make you and/or your client look bad.

Once you’ve got some solid starting designs, you can go nuts with rapid iteration and experimentation.

And if you don’t know why the ad below on the left is godawful, read the next tip.

Tip #2 — Hire a professional

Most people are horrible designers. If you’re not confident in your design skills, hire someone to design a few templates that you can use. To get some good design ideas, browse around online and find well-designed display ads with layouts that could work well as templates.

And if you want a self-serve solution, that’s what my company does (I’ll leave it up to you to find our competitors and figure out why we’re better).

Tip #3 — Use stock photography

At Canned Banners, we see hundreds of display ads designed by amateurs. What’s the number one thing that ruins otherwise good ads? Bad frikkin’ photography. Do not cut corners and make ads using blurry snapshots from your smartphone. Go to inexpensive stock photo websites like, or and buy high-quality photos taken by professional photographers.

Extra Tip-within-a-Tip: Most stock photo websites sell subscriptions. They can be pricey, but if you’re doing high-volume A/B testing, a subscription or package deal is going to be much cheaper than buying images one at a time.

Stock photo websites are also a quick source of design variations that you can test. Running a campaign for a real estate company? Use one ad template and buy 50 good real estate images (the nice house, the “for sale” sign, the happy homeowner couple, the happy agent, etc), throw them in your ads, and see which photos perform best.

That’s enough tips…for now

I could keep going for several more pages, but if you’ve read this far, thanks! I hope these tips give you some food for thought. If you have any questions about display ad design, email me here or follow @cannedbanners on Twitter.


  • iPyxel Creations

    I appreciate what you’re saying here, but beautiful design doesn’t always equate to good ads. Designers think differently than marketers. If you give them a canvas, they will try to fill it with what is pleasing to the eye. That doesn’t always equate to what people click on. I’ve had really amateur looking ads do WAY better than professional looking ads, and evidence of this is throughout performance marketing. The bottom line is your audience really doesn’t care about the design of your ad. They only respond to the message. Now if the feeling they get is irritation, obviously that’s not going to work. But, that doesn’t mean deviants from deviantart can become great marketers.-Tom

  • Myles Younger

    Tom, thanks for the thoughtful comment. Get ready for a loooong response. :-)First, I agree with what you’re saying: display ads do not have to be beautiful in order to be effective, and unsophisticated ad designs can be quite successful (in fact, overly-designed ads can decrease response rates, because the unspoken message of an overly flashy or slick design is "this is an ad, so don’t pay attention to it").But when I say "well-designed," I don’t necessarily mean "pleasing to the eye." A good, well-executed design will convey a great deal of information beyond subjective aesthetics. Take a stop sign, for example. It’s octagonal and red and says "STOP" in big letters in the center. A stop sign isn’t meant to be pleasing to the eye, but it IS a design, and it’s very effective at eliciting an instinctive response from the viewer. That instinct is not accidental; you respond because stop signs are easy to recognize (they don’t look like anything else on the road), you can be punished for disobeying them, and the design has been applied very, very consistently all over the USA for many years.In this same way, even a bare-bones display ad can contain massive amounts of subtle information that draw from a lifetime of experiences and emotions in the viewer’s brain (even the decision to NOT include certain things in your ad is also a form of information for the viewer).And successful bare-bones ads have probably been tested and re-tested by the advertiser. Therefore I would not call the resulting bare-bones design(s) "amateur," because that couldn’t be farther from the truth. The designs were actually carefully selected and iterated by the advertiser, even though the advertiser would not call him/herself a "designer."Often I think the instinct with A/B testing and performance advertising is to chicken out and say "it doesn’t matter that I have no confidence in my own design skills because math and logic will solve the design problem for me." As I tried to demonstrate, you can use math and logic to prove that reliance on PURE iteration is statistically futile, so art, emotion, design, and instinct must therefore play some role.So to rephrase my point: THINK about your display ad designs up front; put some work into them and put some thought into them…don’t just rely on iteration and mathematics alone. If your gut tells you that a basic design will perform better for your campaign, then test a basic design. But make sure that the design you’re using is giving viewers the message you want to send, and if you’re not very good at translating your message into an effective display ad, then find someone who does have that skill.Much of my perspective boils down to this conflict: hackers don’t want to admit that art and emotions play a role in successful ads, and artists don’t want to admit that emotionless logic and mathematics play a role in successful ads. Neither side wants to admit that there a big parts of the equation that are completely outside their direct control, which of course implies that their respective skills are finite and limited, and most people don’t really want to admit that about themselves.Myles Younger

  • way2 profit

    Way2profit provides research based stock market tips in Indian stock market covering NSE, BSE and MCX exchanges.We strongly suggest that if you like to take bit of risk and don???t want to spend too much money and time on new venture then stock market is the right place for you.

  • sharemarketzone

    All stock market tips providers are now at one place and are helping many investors and stock traders. Visit today to take maximum advantage