How to Run A/B Tests on Your Website
When you’re working on a website, you may often rely on your intuition and experience to decide what designs work best. However, using data to make informed decisions is an even better approach. The question is: How can you figure out which design users will prefer?
The answer lies in A/B testing, which is the process of presenting two versions of a web page to visitors and collecting data about which one they prefer. With this information, you’ll be able to create pages you know users will love based on concrete data.
In this article, we’re going to give you a crash course in A/B testing, and explain why you should consider conducting tests on your website. Then we’ll teach you how to A/B test your site, and introduce you to some WordPress plugins that can help. Let’s move over to the lab!
What Is A/B Testing?
A/B (or ‘split’) testing is a process where you take two versions of an element and display each to a random sample of your visitors. The goal is that, with a large enough sample size, you can gain insight into which version your users prefer.
Imagine, for example, that you’re adding a new Call To Action (CTA) to your landing page. You’re not sure what design will work best, but you’ve narrowed the choices down to two options. If you set up an A/B test, you can track which CTA your visitors respond to better, by measuring how many clicks each version gets.
That’s just a basic example. In practice you can use A/B testing to measure all kinds of variables. In fact, A/B testing is something you can benefit from in a broad range of situations, whether it’s for a website, a product design, an email campaign, or something else entirely.
Why Should You A/B Test?
If you’re an experienced enough developer or designer, you might be able to rely on your intuition to determine what designs work best. However, if you really want to know what your users want, running some tests is a smart idea.
Let’s break down exactly how A/B testing can benefit you:
- It can provide you with highly-targeted data about what your visitors prefer.
- You can run successive A/B tests to fine-tune almost every aspect of your website.
- Implementing the results of this testing can help you improve conversions across the board.
What’s more, you can use A/B testing for all kinds of variables. However, there are some limitations to what you can A/B test, which we’ll talk more about in a moment.
What Can You A/B Test?
The good news is that you can get informative results by testing almost any element on your website. You might create variations of your:
- Copy
- Color schemes
- Ads
- Website layouts
- Email subject lines
- Titles
- Forms
- Graphics
- Fonts
- CTAs
- Widgets
- Menus
The possibilities are just about endless. However, there are a few mistakes new A/B testers sometimes make when choosing what to test.
First, you should only test one highly-specific element at a time. For example, don’t change your button’s text and color – only pick one aspect to alter. Changing multiple variables will make it harder to analyze your data, because you can’t be sure which factor influenced users’ behavior.
There is an alternative testing strategy known as multivariate testing, which you can consider using if you want to test multiple elements in combination with one another. However, A/B testing will give you the clearest and most specific results.
The second mistake people sometimes make is focusing only on the big elements, and passing over the small details. For instance, it’s obvious that it can be beneficial to test your button’s text and color. However, you can also get interesting results from testing button styles, shapes, and sizes.
Finally, it’s best to avoid testing new content that you’ve just added to your site. If you published some new copy yesterday, you’ve not going to get accurate results from using it in an A/B test. There’s no way to account for whether users’ behavior is due to the existence of the new element, or the changes in your variations.
How to Run A/B Tests
When you run an A/B test, it helps to think of your website as a lab. If you want to get accurate results, you’ll need to follow strict procedures to make sure the data you gather is accurate. Here’s the approach we suggest you follow.
Step 1: Identify Potential Test Subjects
Technically, you can run an A/B test for almost any variable you want. However, an ideal test subject should meet several criteria:
- The page or element needs to receive a decent amount of traffic.
- There has to be room for improvement, so you can gauge test results.
- You need to be able to create variations of the test subject.
In particular, without enough traffic, the results you get won’t be statistically significant. A few dozen visits doesn’t make for a reliable sample size.
Step 2: Decide on Your Goals
Once you have a subject in mind, you need to set a goal for your test. There are several metrics you might use as your goal, including:
- Clicks
- Page impressions
- Mouseovers
- Bounce rate
For example, let’s say that you intend to run an A/B test for a CTA. In that case, the appropriate metric to track would be clicks. To put it another way, the goal of your test will depend on what its subject is.
Step 3: Choose a Hypothesis
At this point, you have a variable and a metric to test. Now you need to formulate a hypothesis in order to run your experiment.
To elaborate on our earlier CTA example, here a few potential hypotheses you might test:
- Changing the text of my CTA will improve conversions.
- Using a different color for my CTA will result in more clicks.
- Changing the location of my CTA will increase its Click-Through Rate (CTR).
All three of these are perfectly fine hypotheses. However, it’s important to remember that you can only test one of them at a time.
Step 4: Run Your Test and Analyze the Results
Now you can finally put your test into action. To do this, you’ll need to use an A/B testing tool that enables you to show visitors both versions of your test subject (we’ll introduce a few options in a moment).
In our experience, a successful A/B test requires a sample size of at least a few thousand visitors. The smaller your sample pool is, the less reliable your results will be, so you don’t want to rush the process.
Once you have enough data, you can end the test and determine which version performed better according to the goal you set. Then it’s just a matter of implementing that version on your live website and reaping the rewards!
Tools to A/B Test Your WordPress Website
There are a lot of third-party plugins and other tools you can use for A/B testing your website. What follows are some of the best options available.
Nelio AB Testing
The first thing you need to know is that Nelio AB Testing requires you to sign up for a plan in order to use it. If you decide to pay for the plugin, you’ll get access to an A/B testing tool that enables you to run experiments on your pages, posts, and custom post types.
With Nelio AB Testing, you can compare headlines, widgets, CSS, and even entire themes. Plus, you’ll get access to highly-accurate heatmap functionality.
Popup by Supsystic
Whereas our previous pick enables you to test almost any element you want, Popup by Supsystic focuses solely on pop-ups. With this plugin, you can create and A/B test pop-ups that drive conversions. You’ll be happy to know that this plugin is free, and it supports an unlimited number of pop-ups.
WordPress Landing Pages
This is another specialized A/B testing plugin. WordPress Landing Pages provides templates to help you build effective landing pages, and it also has built-in cloning capabilities so you can easily create variations for testing. It supports both single-element and multivariate tests, letting you pursue whichever route makes the most sense for your site.
Instapage
With Instapage, you get tools to build landing pages as well as A/B testing functionality. You can use this plugin’s intuitive builder to put together stunning pages for your site, and then run tests and collect data in your Instapage dashboard. It also provides heatmaps so you can see where your visitors’ attention is drawn, and enables you to track a variety of goals.
Google Optimize
Google Optimize is Google’s official A/B testing platform. If you’re already using Analytics, Search Console, Adwords, and/or PageSpeed Insights, then Optimize is a natural addition to your website management toolbox.
Optimize is free to use, and integrates with Analytics and Adwords to collect user data for your tests. It walks you through the process of setting up those tests in a way that’s easy to understand. Plus, you can now integrate Optimize with your website faster, thanks to the Google Site Kit plugin.
Split Hero
Finally, Split Hero is a WordPress-specific A/B testing service. There’s no plugin to install here, just a range of plans you can sign up for. This tool enables you to create up to four variations using the familiar WordPress editor or your preferred page builder.
Like Optimize, Split Hero walks you through the testing process, so that even less experienced users can easily get used to its features. This platform also compiles your data into an easy-to-read dashboard. Overall, it’s a highly comprehensive solution.
A/B Testing & The WordPress Digital Experience
A/B testing enables you to ‘poll’ your audience without them even knowing. That way, you’ll get access to quantifiable data you can use to improve your pages and provide your users with a better digital experience.
To run a successful A/B test, you’ll need a WordPress web host that provides you with rock-steady performance and support in case you run into any issues. That’s something we can promise on all our plans. Check them out today!
