Set up A/B testing

Estimated duration: 15-30 minutes

This guide demonstrates how to set up A/B testing and the related metrics in your web application using the Prepr GraphQL API.

Introduction

Let's look at an everyday use case. The marketing team wants to increase the clickthrough rate of the landing page header and have prepared two potentially successful headers but are unsure which one will appeal to customers the most. In this case, they run an A/B test.

A/B testing is a simple and efficient way to compare two versions of something to figure out which performs better. In Prepr, marketers can create A and B variants of a section header, then you can render them in your website to show them to visitors randomly. Check out a demo A/B test in action (opens in a new tab).

The image below is an A/B test in the Acme Lease website. It shows the A and B variants of the section header on the Electric Lease landing page.

A/B test on Acme Lease Electric lease landing page

Once implemented, Prepr starts measuring the conversion rate for each variant and determines which is more effective.

Prerequisites

To set up your front end for A/B testing according to the guide below, you need to have the following set up in Prepr:

We will make use of the Landing page Electric Lease content item from the Acme Lease demo data. Take note that the A/B test is defined in a Stack field. Check out the marketers docs for more details on how to add an A/B test to a content item.

Landing page Electric Lease A/B test

Set up your front end for A/B testing

Before setting up your front end for A/B testing, make sure the following code has been prepared:

Add the Prepr Tracking Code to track visitor events.
Get a customer Id to identify visitors. (opens in a new tab) Prepr uses this customer Id to match a visitor to an A or B variant.

The code above uses the Prepr customer Id. For your project, you could use your own customer Id.

Now you're ready to do the A/B test setup with the below steps.

Retrieve variants using the API

Now it's time to retrieve the Landing page Electric Lease content item including the section header variants. Check out the marketers A/B testing guide for more details on how this A/B test was created in Prepr.

Copy the sample code below into your front-end application.

query{
  Page (id: "8da86294-c15d-415a-9fa3-6ea289c9403c") { 
    _id
    title
    _slug
    stack { 
      __typename
      ... on SectionHeader {
        _id
        _context {
          variant_key
          kind
          variant_id
        }
        title
        cta_button {
          text
        }
      }
    }
  }
}

Update the id string value of the query with the Content item ID of the Landing page Electric Lease. See the image below on where to find the Content item ID on the content item page.

content item ID

To retrieve the page content, call the API using the query above and pass a Prepr Customer ID as a header. The Prepr Customer ID is required to determine which variant a visitor gets, so set the Prepr Customer ID header value to the customer Id that you got earlier (opens in a new tab).

--header 'Prepr-Customer-Id: <YOUR-CUSTOMER-ID-VARIABLE>'

Follow the Connecting front-end apps guide to learn more about retrieving content items for a specific framework.

As a response to your API request, Prepr sends the variant to be displayed. And your app then needs to build its response based on this variant. See an example of a response below:

{
  "data": {
    "Page": {
      "_id": "8da86294-c15d-415a-9fa3-6ea289c9403c",
      "title": "Landing page Electric Lease",
      "_slug": "electric-lease",
      "stack": [
        {
          "_context": {
            "kind": "AB_TEST",
            "variant_key": "eyJpIjoiYmU3OTkxMWEtNjE3My00YmYzLWE5ZGUtZTg2YzYzNWMyYjhiIiwiZSI6IjVmMWFhM2I4LTQyZjktNGY2OS04MzYzLTkxMDFhZmFlNDQ2ZiIsInYiOiJBIn0=",
            "variant_id": "A"
          },
          "title": "Short. Affordable. Electric.",
          "cta_button": {
            "text": "Find your car"
          }
        }
      ]
    }
  }
}

The algorithm remembers which visitor gets which variant so that the same visitor always gets the same section header variant when interacting with your web app.

Take note of the following important context fields:

kind - This returns a value of AB_TEST when an A/B test is enabled for this element in the Stack.
variant_id - This returns a value A or B to indicate the returned variant of the A/B test.
variant_key = A unique Id for the variant. You will use this field value in the next step to indicate which components need to be tracked for impressions and clicks.

You can do a test run of the A/B test in Prepr before launching it on your live website:

Open the API Explorer as described here.
Copy and paste the Graphql query above into the Operations pane.
Under the Header section, enter Prepr-Customer-Id with any customer Id value. On the Segments screen, find the Id on the right when you edit a specific customer or create one manually if you don't have any customers.
Run the query.
Re-run the query for a different Customer ID.

The algorithm should match a Customer ID and a section header variant. For example, Customer 1 always gets Variant A, Customer 2 - Variant B.

Add HTML attributes to track impressions and clicks

To track visitor impressions and clicks for an A/B test and get accurate metrics, you need to add HTML attributes to elements in your A/B test. Usually, you'd collaborate with the marketing team to choose the element that indicates a conversion in the A/B test.

In our example, we have a section header with some specific text and a call-to-action button for each variant, so we want to report on the following types of visitor interactions:

Impressions: How many visitors looked at the section header for each variant.
Conversions: How many visitors clicked the CTA button for each variant.

Prepr will determine the conversion rate and other related metrics based on these events and will show you which variant performed best.

To start collecting impressions and clicks in Prepr, add the HTML attribute data-prepr-variant-key to the Section header and the HTML attribute data-prepr-variant-event to the CTA button.

The impression is recorded when an element, for example, the Section header, scrolls into view. While the click is recorded when an element, like the CTA button is clicked.

Set the data-prepr-variant-key value to the Prepr variant key value returned in the query response.

See an example code snippet to include the HTML attributes below:

...
<!-- section header -->
<div class="..."  data-prepr-variant-key="{{ variant_key }}">
  <!-- section header image -->
  <img src="..." class="...">
  <div class="...">
    <div class="...">
      <!-- section header text -->
      <h1 class="...">Short. Affordable. Electric.</h1>
      <span>
        <!-- section header button -->
        <a href="/catalog?category=electric" data-prepr-variant-event>
          <button class="...">Find your car</button></a>
      </span>
    </div>
</div> 
...

That’s it. You've set up A/B testing in your front end. A correct section header variant will be displayed every time a customer visits your website and Prepr will calculate metrics for the A/B test.

As visitors interact with your web app, data will be collected and metrics will be visible in the A/B test. Go to the Landing page Electric Lease content item to see the metrics.

Metrics example

Congratulations, you have successfully set up A/B testing in your front end. Check out the next section if you are implementing a static website.

Implementing A/B testing for static (SSG) websites

Unlike dynamic sites, static sites serve pre-built content to a web browser without calls to a database. To perform A/B testing for static site rendering, you need to set up variants ahead of time (particularly before you build the web app) and then split traffic between different routes using the edge middleware (see an example tool (opens in a new tab)). Let’s see it in detail.

The A/B testing for static site rendering looks like this:

On pages where an A/B test needs to be triggered, you call the API to pre-fetch variants. Your request must include the following fields:

_context - system field that contains additional details about variants such as the following fields:
- kind – returns a value of PERSONALIZATION or AB_TEST.
- variant - returns a value of A or B if kind has a value of AB_TEST

Here’s an example code snippet. Update the id string of the query with your Content item ID (find the Id in the right-hand column on the Page content item in Prepr).

query{
	Page (id: "8da86294-c15d-415a-9fa3-6ea289c9403c") { 
		_id
		title
		_slug
		stack (personalize: false) { 
			__typename
    	... on SectionHeader {
      	_id
      	heading
      	text
      	cta_label
				_context {
					kind
					variant_id
				}
			}
  	}
	}
}

When a user visits your web app, you make a one-time request to know if this user is assigned to Bucket A or Bucket B. Your request must include the following headers:

Prepr-Customer-Id with a session Id of a visitor
Prepr-Bucket-Customer = true

The bucketing runs at the edge location of the CDN and takes no longer than 7 ms. A response to your API request contains a header – either X-Prepr-Customer-Bucket A or X-Prepr-Customer-Bucket B.

See an example request below:

curl --location --globoff 'https://graphql.prepr.io/<YOUR-ACCESS-TOKEN>' \
--header 'Prepr-Customer-Id: 409ad1af-d644-4d3f-8cb9-691c0318a980' \
--header 'Prepr-Bucket-Customer: true'

With this information, the front-end middleware can redirect a visitor to a respective variant of your A/B test. For all the requests from the same visitor, the same variant will be shown.

Want to learn more?

Check out the following guides:

Schedule a free consultation

Do you want to get started with A/B testing but still have questions or want a demo?

Schedule a free call (opens in a new tab) with a Prepr solution engineer.

Set up personalization Set up recommendations

Was this article helpful?

We’d love to learn from your feedback