Set up A/B testing

Set up A/B testing

Estimated duration: 30-45 minutes

Easily add A/B testing to your web application using Prepr. This tutorial demonstrates how to set up A/B testing using the Prepr GraphQL API.

Introduction

Let's look at an everyday use case: the marketing team is about to run a new promotion on the website and have prepared two potentially successful promotions but are unsure which one will appeal to customers the most. In this case, they can run A/B tests.

A/B testing is a simple and efficient way to compare two versions of something to figure out which performs better. In Prepr, marketers can create A and B variants of a page header, one for each promotion, then you can render them in your website to show them to visitors randomly. Check out a demo A/B test in action (opens in a new tab) like in the image below.

A/B testing introduction (opens in a new tab)

Once implemented, marketers can use an analytics tool like Google Analytics to measure the click-through rate for each page header version and determine which is more effective.

To set up A/B testing, you need to complete the following steps:

Step 1: Add the A/B test to the front end. This is a one-time update to the code which means that any updates to A/B test content will automatically be applicable without further code changes.

Step 2: Connect analytics to the front end. While the marketing team will set up analytics, you'll need to collaborate with them to include necessary analytics tracking in the front end.

Prerequisites

To set up A/B testing in the front end according to the guide below, you need to have the following set up in Prepr:

We will make use of the Home page with A/B testing content item from the demo data. Take note that the A/B test is defined in a Stack field. Check out the marketers docs for more details on how to create an A/B test in a Stack field.

Home page A/B test

Step 1: Add the A/B test to the front end

In this step we look at how to do the following:

  • Get a customer Id to identify visitors
  • Retrieve the variants using the API
  • Add HTML attributes to the page header component

Get a customer Id to identify visitors

When running an A/B test we have to indicate who is visiting the website to make sure that visitors get the same variant each time they visit the page. Prepr uses a customer Id to match a visitor to an A or B variant. You can use your own customer Id or an existing Prepr customer Id. Check out the Next complete starter GitHub repo (opens in a new tab) for an A/B test example that uses a UUID cookie for the customer Id.

If you are not keeping track of customer Ids on your site, we recommend using the session cookie to identify your visitors. Make the session persistent by setting the cookie's lifetime parameter to something like 90 days. This means that you get the same session Id even when a customer visits the site multiple times. Get the session Id in your front end and use this value as the customer Id to be sent to Prepr. Prepr will then match each variant to this Id.

Retrieve variants using the API

Now it's time to retrieve the Home page with A/B testing content item including the page header variants. Check out the marketers A/B testing guide for more details on how this A/B test was created in Prepr.

Copy the sample code below into your front-end application.

query{
  Page (id: "8da86294-c15d-415a-9fa3-6ea289c9403c") { 
    _id
    title
    _slug
    stack { 
      __typename
      ... on PageHeader {
        _id
        heading
        text
        cta_label
        _context {
          kind
          variant_id
          group_id
        }
      }
    }
  }
}

Update the id string value of the query with the Content item ID of the Home page with A/B testing. See the image below on where to find the Content item ID on the content item page.

content item ID

To retrieve the page content, call the API using the query above and pass a Prepr Customer ID as a header. The Prepr Customer ID is required to determine which variant a visitor gets. Get a customer Id as shown above and use this value to set the Prepr Customer ID header value. See an example in the below code snippet:

--header 'Prepr-Customer-Id: 409ad1af-d644-4d3f-8cb9-691c0318a980' 

We recommend that you always pass the Prepr Customer ID to retrieve any content with a stack whether an A/B test has been created in Prepr or not. This way, when A/B tests on any items or components are created in Prepr later, these will automatically be activated from the front end.

Follow the Connecting front-end apps guide to learn more about retrieving content items for a specific framework.

As a response to your API request, Prepr sends the variant to be displayed. And your app then needs to build its response based on this variant. See an example of a response below:

{
  "data": {
    "Page": {
      "_id": "8da86294-c15d-415a-9fa3-6ea289c9403c",
      "title": "Home page",
      "_slug": "home-page",
      "stack": [
        {
          "__typename": "PageHeader",
          "heading": "Bring a friend to any baking course for free",
          "text": "Sign up for any baking course in our calendar together with a friend and they can join for free!",
          "cta_label": "Sign up now!",
          "_context": {
            "kind": "AB_TEST",
            "variant_id": "A",
            "group_id": "34022364-7ed3-46ff-9055-a208c5b0cccd"
          }
        }
      ]
    }
  }
}

The algorithm remembers which visitor gets which variant so that the same visitor always gets the same page header variant when interacting with your web app.

You can do a test run of the A/B test in Prepr before launching it on your live website:

  1. Open the API Explorer as described here.
  2. Copy and paste the Graphql query above into the Operations pane.
  3. Under the Header section, enter Prepr-Customer-Id with any customer Id value. On the Segments screen, find the Id on the right when you edit a specific customer or create one manually if you don't have any customers.
  4. Run the query.
  5. Re-run the query for a different Customer ID.

The algorithm should match a Customer ID and a page header variant. For example, Customer 1 always gets Variant A, Customer 2 - Variant B.

Add HTML attributes to the page header component

To track visitor interactions for A/B testing and get accurate reporting results, we need to add attributes to the page header. In our example, we have a Page header with some specific text and a call to action link for each variant, so we want to report on the following types of visitor interactions:

  • Impressions: How many visitors looked at the page header for each variant.
  • Clicks: How many visitors clicked the CTA link for each variant.

We can determine the click-through rate based on these two events, which will show which variant performed best. Because the above interactions need to be per variant, we also need a custom variable to identify the variant.

See an example code snippet to include the HTML attributes below:

<!-- Page header -->
<div
  id="{{__typename}}_{{group_id}}"
  prepr-variant="{{ variant_id }}"
  ...
>
<!-- Call to action link in the Page header-->
  <a
    href="#" 
    id="cta_link_{{__typename}}_{{group_id}}"
    prepr-variant="{{ variant_id }}"
    ...
  >Learn more</a>
</div>

In the above example, we set the Ids dynamically by using the following fields returned in the query response:

  • __typename - The element type, for example, PageHeader.
  • group_id - A unique Id to identify a specific A/B test group. This is useful to identify all elements that have an A/B test enabled.
  • variant_id - This returns a value of A or B.

Using the query response above as an example the Ids get the following values:

  • id = "PageHeader_34022364-7ed3-46ff-9055-a208c5b0cccd"
  • prepr-variant = "A"

That’s it. You have integrated the A/B test into your website. A correct page header variant will be displayed every time a customer visits your website.

The next step is to set up the analytics to measure the click-through rate to see which variant performs best.

Step 2: Connect analytics to the front end

Prepr A/B testing works with any analytics tool. If you already have your analytics tool set up to measure visitor interactions on your web app, then you can ignore this step.

  1. If not already done, work with the marketing team to connect GA4 and GTM to your web app by following the GA4 and GTM setup process (opens in a new tab).

Once completed, the analytics tool is connected to your front end. The rest of the analytics setup is ordinarily done by the marketing team and you can find more details in the Run A/B tests guide for marketers.

After some time the marketing team will be able to use the reports to determine the better performing variant and close the A/B test. This means that Prepr will deliver the chosen variant to your website visitors. The other variant will be removed from the Stack.

Congratulations, you have successfully set up A/B testing in your front end. Check out the next section if you are implementing a static website.

Implementing A/B testing for static (SSG) websites

Unlike dynamic sites, static sites serve pre-built content to a web browser without calls to a database. To perform A/B testing for static site rendering, you need to set up variants ahead of time (particularly before you build the web app) and then split traffic between different routes using the edge middleware (see an example tool (opens in a new tab)). Let’s see it in detail.

The A/B testing for static site rendering looks like this:

  1. On pages where an A/B test needs to be triggered, you call the API to pre-fetch variants. Your request must include the following fields:
  • _context - system field that contains additional details about variants such as the following fields:
    • kind – returns a value of PERSONALIZATION or AB_TEST.
    • variant - returns a value of A or B if kind has a value of AB_TEST

Here’s an example code snippet. Update the id string of the query with your Content item ID (find the Id in the right-hand column on the Page content item in Prepr).

query{
	Page (id: "8da86294-c15d-415a-9fa3-6ea289c9403c") { 
		_id
		title
		_slug
		stack (personalize: false) { 
			__typename
    	... on PageHeader {
      	_id
      	heading
      	text
      	cta_label
				_context {
					kind
					variant_id
				}
			}
  	}
	}
}
  1. When a user visits your web app, you make a one-time request to know if this user is assigned to Bucket A or Bucket B. Your request must include the following headers:
  • Prepr-Customer-Id with a session Id of a visitor
  • Prepr-Bucket-Customer = true

The bucketing runs at the edge location of the CDN and takes no longer than 7 ms. A response to your API request contains a header – either X-Prepr-Customer-Bucket A or X-Prepr-Customer-Bucket B.

See an example request below:

curl --location --globoff 'https://graphql.prepr.io/<YOUR-ACCESS-TOKEN>' \
--header 'Prepr-Customer-Id: 409ad1af-d644-4d3f-8cb9-691c0318a980' \
--header 'Prepr-Bucket-Customer: true'
  1. With this information, the front-end middleware can redirect a visitor to a respective variant of your A/B test. For all the requests from the same visitor, the same variant will be shown.

Want to learn more?

Check out the following guides:

Schedule a free consultation

Do you want to get started with A/B testing but still have questions or want a demo?

Schedule a free call (opens in a new tab) with a Prepr solution engineer.

Was this article helpful?

We’d love to learn from your feedback