What Is A/B Testing And Why It Is Essential For Your Online Campaigns

A/B…? Come again?

A/B, or split testing, is one very important part in online marketing and should be practiced by every company doing business or marketing online. But what exactly is this A/B testing? Behind the term stands the basic notion of testing different variations of an assumption and see which one performs best. It’s that simple, but really effective.

I think it is almost obvious why you should split test: It’s simply a great way to optimize your online campaigns. By changing just minor elements, you’re able to let your conversion rate hit the roof. Basically, A/B tests can be used in all online marketing areas. Here, I’ll tap into online advertising, email marketing and website optimization.

The Process of A/B Testing

The idea of split testing is to take an existing element or situation and make it your baseline control. Then, according to a predefined hypothesis, you create variations and test them against your control element. If one beats your current element, replace it with your “winner”. And then what? Well, you do the whole process again and again, using the winning variation as your new baseline control. Let’s go over all that step-by-step:

  1. Come up with a (testable) question. For example: “Does a headline highlighting my incentive (discount, etc) generate more leads/customers?”
  2. Background research. With this question in mind, try to find out how your website visitors or target audience behaves. This way you can make more sense out of your findings later on.
  3. Hypothesis: Based on the previous steps, formulate an assumption. For example: “A headline highlighting an incentive rather than the product benefits (discount, etc) generates more leads.”
  4. Determine your required sample size and test duration.
  5. Test your assumption. Create one or several (but not too many) variations of your control element (e.g. headline) and start A/B testing.
  6. Analyze the data. Measure all important variables, such as generated leads, and draw a conclusion.
  7. Make adjustments accordingly. If your findings point out that one variation outperformed the control unit, then adjust/replace it.
  8. Make your new element the control unit and start the process again.

A/B Testing in Online Advertising

Split testing in online advertising is a common practice and virtually all online marketers test their ads – ALL of them. Also, all major ad service providers, such as AdWords and Facebook Ads, support split tests. This makes split testing ad copy and images a piece of cake.


Test your ad headlines, copy and, if possible, ad extensions as well as images. Just keep one important thing in mind: Only change one element! Don’t test an ad with a changed headline AND text, because then you have to guess which of the two is responsible for performance changes. Guessing doesn’t bring you anywhere.

A/B Testing in Email Marketing

Email marketers also split test their client emails and newsletters all the time. Luckily it’s a very straightforward procedure. You can test subject lines, writing style, time of day/week, images, address of sender and HTML vs. plain text. Again, only use variations with one element. A/B testing is a learning process, in which you find out step-by-step which works best for you.

The most important variables to track here are:

  • Open rate
  • Click through rate
  • Unsubscribes

Email marketing provider usually offer split testing. One of the most popular provider, Mailchimp, even gives you the option to automate your tests. After testing a number of email variations on a small part of your email list and determining a winner, Mailchimp can automatically send it out to the rest of the list. Quite convenient, isn’t it?


A/B Testing Your Website

One quite obvious thing to split test is your website itself. Since it’s representing your business and ultimately responsible to turn visitors into leads or customers, it should not only look great, but also be as perfectly optimized as it can be.

Optimizing a website doesn’t just stop at testing a headline or text. It goes as far as split testing major parts of it, such as the navigation bar, positioning of your logo, call-to-action buttons and coloring of every remotely important element. Of course, before testing rather exotic things, such as footer links, you should first focus on your site’s headlines, call-to-actions, registration forms, images and security badges.

To get more info on website optimization, check out our website optimization guide.

Last Words

I’ve only scratched the surface of what’s possible with A/B testing. So, there’s a lot more to discover and to learn. I recommend checking out Optimizely and Visual Website Optimizer. They have great resources on A/B testing as well as great tools to easily create A/B tests using a sort of WYSIWYG editor. Their service cost a relatively small monthly fee, but, compared to the potential gains in conversions, it’s a very good deal.

What are your biggest A/B test success stories? Please share them in the comments below.

Comments are closed.