Run A/B tests

Easily add A/B testing to your web application using Prepr. This guide shows you how to run A/B tests in Prepr CMS.

Introduction

Let's look at an everyday use case: you are about to run a new promotion on your website and have prepared two potentially successful promotions but are unsure which one will appeal to your customers the most. In this case, you can run an A/B test.

A/B testing is a simple and efficient way to compare two versions of something to figure out which performs better. In Prepr, you can create A and B variants of a page header, one for each promotion, then show them to your website visitors randomly. Check out a demo A/B test in action like in the image below.

After setting up an A/B test, you can use an analytics tool like Google Analytics to measure the click-through rate for each page header version and determine which is more effective.

To run A/B tests, you need to complete the following steps:

Step 1: Create an A/B test in Prepr. In this step, we show you how to create an A/B test with the variants that you want to show to your web app visitors.

Step 2: Get the HTML attributes for analytics. Get the HTML attributes of the A/B test elements from the front end.

Step 3: Set up analytics. Using your preferred analytics tool, you will measure and compare click-through-rates for each of the variants.

Step 4: Evaluate and close the A/B test. Once you know which variant performed better, we show you how to select it and end the A/B test.

Prerequisites

To run A/B tests based on this guide, you need to have the following set up in Prepr:

• A Prepr CMS account
• An environment with demo data in Prepr

We will make use of the Home Page content item from the demo data which is based on the Page pattern.

Step 1: Create an A/B test in Prepr

Go to the Content tab, click the Home page content item and add the A/B test as follows:

1. To start an A/B test, simply click the A/B test icon on the page header. When you do this, a B variant will be created, which is a copy of the A variant.
2. Update the images and text for each variant of the promotion.

That’s it. You’ve just set up the A/B test in Prepr.

Once you’re ready, click Publish and close.

Step 2: Get the HTML attributes for analytics

If not already done, a front developer needs to apply a one-time setup for A/B testing in the front end as explained in the A/B testing setup guide for developers. Once A/B testing is enabled in the front end, a correct variant will be displayed every time a customer visits the website according to your A/B test content. You can see either the A or B version by visiting the page from different devices or by switching to incognito mode when browsing.

From your browser, visit the page that was set up with the A/B test and right-click the A/B test element. Select Inspect to get the HTML attribute value for the id of the A/B test element and get the name of the variant HTML attribute like in the example below. Do the same for the id of the click element like a call to action link that you want to use to measure the CTR. You need to reference these attribute names accurately in the analytics tool in the next step to measure which variant performs best.

<section id="PageHeader_b997e121-5a8a-4527-a4f5-431029bdf" 
prepr-variant="A"  ...>
  <div>
    ...
    <a id="cta_link_PageHeader_b997e121-5a8a-4527-a4f5-431029bdf" href="#" ... >...</a>
  </div>
</section>

Step: 3 Set up analytics

Prepr A/B testing works with any analytics tool. If you already have your analytics tool set up to measure visitor interactions on your web app, please adapt the following explanation to your specific case. In this example we show you how to measure visitor interactions on the A and B variants using GA4 and GTM.

Set up GTM

1. If not already done, work together with the front-end developer to connect GA4 and GTM to the web app by following the GA4 and GTM setup process.

2. In GTM, create a custom variable and set the Attribute name to the name of the variant HTML attribute that you found in the previous step.

Create impression and click events in GTM

1. Create tags in GTM for the following events:

• An element visibility event that fires when the page header section is 100% visible to the visitor.

Make sure that the visibility trigger as shown in the image below has the same Element ID value as the HTML attribute id value of the A/B test element that you found in the previous step.

• A click link event that fires when a user clicks the CTA link.

Make sure that the click trigger as shown in the image below has the same Click ID value as the HTML attribute id value of the CTA element that you found in the previous step.

2. Test your events by clicking on the Preview button and enter the URL of the web app that you want to test.

3. If the tags are firing successfully and the variables are being populated as expected from the front end, click the Submit button to publish these tags to your web app.

Create reports in GA4

1. Create a custom dimension in GA4 so that the variant parameter is available for detailed reporting in Google Analytics.

2. Create a Funnel exploration in GA4. Click on the + button for DIMENSIONS to add the custom dimension A/B test variant. Add STEPS for the visibility and click events and specify a BREAKDOWN by the custom dimension.

While the A/B test is active and users visit each variant, the reporting in your analytics tool will eventually show significant results on the variants.

Step 4: Evaluate and close the A/B test

The last step is to check the CTR for both variants and if the outcome is already significant, you can choose the winner and end the experiment.

Use this online A/B test calculator to check if the collected data is significant enough to determine a winning variant as follows:

1. From your report created in the previous step, copy the number of users from the visibility event from your report to fill in the Visitors A and Visitor B totals.
2. Copy the number of users who completed the click event in the Conversions A and Conversions B fields.
3. After you apply the changes, this calculator indicates if your result is significant or not.

When you get a significant result, you can then choose the winning variant in Prepr by following the steps

1. Go to the content item and click the End A/B test button.
2. Click Keep A or Keep B to choose the best performing variant.

From now on, Prepr will deliver the chosen variant to your website visitors. The other variant will be removed from the Stack.

Congratulations, you have successfully implemented an A/B test and used it to improve the customer experience on your web app.

Want to learn more?

Check out the following guides:

• Collect event data
• Manage segments
• Personalize website

Schedule a free consultation

Do you want to get started with A/B testing but still have questions or want a demo? Book a free appointment with a Prepr solution engineer right away.