A guide to A/B testing that improves your ecommerce conversion rate [with examples]

An effective A/B test is like a puzzle. 

When employed correctly, the many pieces fit together and create a clear picture of something completely new. 

But, if even one piece to the puzzle is missing, the story can be indecipherable. 

That's why we put together this guide to A/B testing that improves your conversion rate. After over a decade of running tests for ecommerce brands big and small, we have all the details you need to consider for executing great tests.

We'll cover: 

  • What is A/B testing? 
  • Step-by-step how to execute an A/B test
  • A/B testing examples and ideas

What is A/B testing? 

A/B testing is one of the most efficient methods for increasing revenue and optimizing the customer experience on your ecommerce site. 

It is a simple and iterative way to test hypotheses by comparing two versions of something to find out which one performs better. For ecommerce websites, an A/B test typically means showing half your visitors one webpage variant, and half another. 

You determine the effectiveness of each variant by observing visitor actions. The variant that prompts them to do what you want them to do most often (buy something, sign up for something, click a link, etc.) is the winner. 

On your website, A/B testing can be used to experiment with variations of content, design, or navigation; gather concrete evidence; and make decisions based on real data. With this data you can optimize your website and continue experimenting. Each test can lead to a slightly improved conversion rate or customer experience.

That process of A/B testing one variation of a web page against another can continue indefinitely. There's always room for improvement and new items to test. A/B testing is a direct method of field-testing your ideas.

As Jeff Bezos says, "Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day." 

The more A/B tests you perform, the more you can optimize your site for conversions.

Step-by-step how to execute an A/B test

I encourage you to approach conversion optimization and A/B testing scientifically. Rely on the data to guide you, rather than trying to guide the data.

Here are the steps to conduct testing on your ecommerce or lead-generation website:

  1. Identify clear objectives for your testing program. Think about it as a problem you wish to address or something you want to understand.
  2. Establish website challenges through both qualitative and quantitative research. Research is conducted to set baselines, surface patterns, and customer sticking points.
  3. Isolate testing opportunities. Stay user-focused and try to pick website challenges that are customer-centric rather than product-focused.
  4. Formulate an A/B test hypothesis. Use your current knowledge to propose a solution or explanation for the problem.
  5. Prioritize your testing opportunities. To avoid decision paralysis, organize your tests by funnel point (or page) and select a few particularly exciting opportunities across various points of the conversion funnel. 
  6. Set up an experiment to test your hypothesis. Here you will design your test. Formalizing this pre-development review is a valuable way to improve your testing skills and keep the visual design aligned with the brand.
  7. Run the test and analyze the results. 
  8. Learn from the test, and keep testing until you're satisfied with the results

For teams just starting out, try to focus on 1-3 testing objectives. Rank those based on priority to begin your roadmap. 

For example, increasing conversion rates might be more important to your team than improving email signups. So, knowing where to put your time and attention and aligning across the testing team (and other key business stakeholders) is a non-negotiable first step.   

Follow that path we laid out above, and you will learn. Deviate, and you can head down the wrong trail. That will waste time and likely cost you precious budget. 

And if you are thinking of running multiple A/B tests at the same time, make sure you learn how to do so without compromising results. 

A/B testing ideas and examples

Now you know the basics of how to run an A/B test, but what does this look like in action? 

Below you'll find a few real-life A/B testing examples and ideas to get you brainstorming for your own testing and optimization process. 

Mobile test ideas & example

During the research phase, it often becomes clear how crucial it is to A/B test a mobile experience. After all, by 2024, trusted research company eMarketer expects global retail mobile commerce sales to reach nearly $4.5 trillion, making up 69.9% of total retail ecommerce sales. You want your mobile experience to be as seamless as possible.

So, here are a few ideas to get you started: 

  • Experiment with unique mobile navigation to see if your visitors browse differently on desktop vs mobile.
  • Try out some icons to convey themes or ideas in a concise way. Observe if this impacts user comprehension.
  • Test the length of your headlines and subheadlines to see how shoppers react to each.

Wondering what this might look like in real life? Here's an example. 

Rollie Nation is an Australian footwear brand known for its playful yet sophisticated travel shoes. Combing through the brand's data, the team discovered that their total website traffic was made up of nearly two-thirds mobile visitors, but mobile visitors were 3 times less likely to convert than desktop users within a session.

This large disparity indicated a problem. To promote more intentional browsing patterns on mobile devices, the placement of the mobile menu was moved from the bottom of the screen to the top.

Conversion mobile test ideas & example

After one month and over 80,000 experiment sessions, this Mobile Menu test variant produced a 69% improvement of clicks on the menu, and a 6% higher conversion rate than the control, resulting in a return on investment for Rollie Nation of 165%. Keep in mind, these results are only from 1 month of testing!

Category page test ideas & example

Your category page is key to taking shoppers further down the conversion funnel. And if your research identifies this as a sticking point for customers, you are definitely going to want to run some A/B tests. Here are some ideas to get your creative juices flowing:

  • Explore different category naming conventions in case a new variation is easier for customers to understand (or more appealing). 
  • Try personalizing the content or categories a user sees to improve their on-site experience.
  • Experiment with different images, maybe testing product photography vs lifestyle.

Now let's look at a category page A/B test in action. 

Take this example from the customizable office furniture brand, Fully. Research from heatmaps and session recordings indicated high engagement with the filters on the product category pages, specifically on the different types of desks. 

The hypothesis was formed that users not already familiar with Fully's product offering may not know the difference between different categories, or what desk might best suit their needs. The test and control are pictured below, and this individual test produced a 75:1 return on investment for the brand.

Category page control and variant

Post-purchase A/B tests

Pre-purchase is crucial, but another great place to test ideas for your website is during the post-purchase experience. 

Post-purchase offers are displayed immediately after the checkout and before the thank you page, when customer loyalty is high. On average, merchants using CartHook see a 15% acceptance rate on offers! A/B testing your post-purchase upsells will help you reach your highest potential conversion rate. 

Built-in testing logic allows brands to see:

  • Which post-purchase offers are performing best
  • Metrics that clearly determine which offer set performed better, including the second offer that's triggered by accept/decline logic 
  • Total visits, conversion rates, revenue, and revenue per visitor for each offer and offer set
  • Reports on individual offers and set analytics

Try CartHook free for 14-days and take a video tour of post-purchase A/B testing functionality.

If you're looking for more ideas, here's a collection of 50+ ideas and examples for your A/B test

Learn from your A/B testing

Your A/B tests only fail to work when they aren't set up properly or you refuse to learn from the data they provide. Today's digital marketing and ecommerce managers are bombarded with information, so it's easy to push it all away and return to intuitive decision-making.

The problem is that your past experience doesn't always accurately describe the future. Things change. And in the world of online marketing, things change rapidly. 

Once you've set up a reasonable testing calendar and protocol, you'll get more comfortable with drawing on test results to help inform your decisions. In turn, you will build a better, stronger ecommerce website. 

If you need help as you get started or simply want to talk about your plans with a team that tests daily… contact The Good or review our services.

We love this stuff!

Want to see more content like this?

Thanks, you're all signed up!
Oops! Something went wrong while submitting the form.

Join the world's fastest-growing indie brands.

Request a Demo