Set up A/B testing

Estimated duration: 30-45 minutes

Easily add A/B testing to your web application using Prepr. This tutorial demonstrates how to set up A/B testing using the Prepr GraphQL API.

Introduction

Let's look at an everyday use case: the marketing team is about to run a new promotion on the website and have prepared two potentially successful promotions but are unsure which one will appeal to customers the most. In this case, they can run A/B tests.

A/B testing is a simple and efficient way to compare two versions of something to figure out which performs better. In Prepr, marketers can create A and B variants of a page header, one for each promotion, then you can render them in your website to show them to visitors randomly. Check out a demo A/B test in action like in the image below.

Once implemented, marketers can use an analytics tool like Google Analytics to measure the click-through rate for each page header version and determine which is more effective.

To set up A/B testing, you need to complete the following steps:

Step 1: Add the A/B test to the front end. This is a one-time update to the code which means that any updates to A/B test content will automatically be applicable without further code changes.

Step 2: Connect analytics to the front end. While the marketing team will set up analytics, you'll need to collaborate with them to include necessary analytics tracking in the front end.

Prerequisites

To set up A/B testing in the front end according to the guide below, you need to have the following set up in Prepr:

• A Prepr CMS account
• An environment with demo data in Prepr

We will make use of the Home page with A/B testing content item from the demo data. Take note that the A/B test is defined in a Stack field. Check out the marketers docs for more details on how to create an A/B test in a Stack field.

Step 1: Add the A/B test to the front end

In this step we look at how to do the following:

• Get a customer Id to identify visitors
• Retrieve the variants using the API
• Add HTML attributes to the page header component

Get a customer Id to identify visitors

When running an A/B test we have to indicate who is visiting the website to make sure that visitors get the same variant each time they visit the page. Prepr uses a customer Id to match a visitor to an A or B variant. You can use your own customer Id or an existing Prepr customer Id. Check out the Next complete starter GitHub repo for an A/B test example that uses a UUID cookie for the customer Id.

Retrieve variants using the API

Now it's time to retrieve the Home page with A/B testing content item including the page header variants. Check out the marketers A/B testing guide for more details on how this A/B test was created in Prepr.

Copy the sample code below into your front-end application.

query{
  Page (id: "8da86294-c15d-415a-9fa3-6ea289c9403c") { 
    _id
    title
    _slug
    stack { 
      __typename
      ... on PageHeader {
        _id
        heading
        text
        cta_label
        _context {
          kind
          variant_id
          group_id
        }
      }
    }
  }
}

curl --location --request POST 'https://graphql.prepr.io/<YOUR-ACCESS-TOKEN>' \
--header 'Prepr-Customer-Id: 409ad1af-d644-4d3f-8cb9-691c0318a980' \
--header 'Content-Type: application/json' \
--data '{"query":"query{\n  Page (id: \"8da86294-c15d-415a-9fa3-6ea289c9403c\") { \n    _id\n    title\n    _slug\n    stack { \n        __typename\n    ... on PageHeader {\n        _id\n        heading\n        text\n        cta_label\n        _context {\n                kind\n                variant_id\n            }\n        }\n    }\n  }\n}","variables":{}}'

var myHeaders = new Headers();
myHeaders.append("Prepr-Customer-Id", "409ad1af-d644-4d3f-8cb9-691c0318a980");
myHeaders.append("Content-Type", "application/json");

var graphql = JSON.stringify({
  query: "query{\n  Page (id: \"8da86294-c15d-415a-9fa3-6ea289c9403c\") { \n    _id\n    title\n    _slug\n    stack { \n        __typename\n    ... on PageHeader {\n        _id\n        heading\n        text\n        cta_label\n        _context {\n                kind\n                variant_id\n            }\n        }\n    }\n  }\n}",
  variables: {}
})
var requestOptions = {
  method: 'POST',
  headers: myHeaders,
  body: graphql,
  redirect: 'follow'
};

fetch("https://graphql.prepr.io/<YOUR-ACCESS-TOKEN>", requestOptions)
  .then(response => response.text())
  .then(result => console.log(result))
  .catch(error => console.log('error', error));

// WARNING: For POST requests, body is set to null by browsers.
var data = JSON.stringify({
query: "query{\n  Page (id: \"8da86294-c15d-415a-9fa3-6ea289c9403c\") { \n    _id\n    title\n    _slug\n    stack { \n        __typename\n    ... on PageHeader {\n        _id\n        heading\n        text\n        cta_label\n        _context {\n                kind\n                variant_id\n            }\n        }\n    }\n  }\n}",
variables: {}
});

var xhr = new XMLHttpRequest();
xhr.withCredentials = true;

xhr.addEventListener("readystatechange", function() {
if(this.readyState === 4) {
  console.log(this.responseText);
}
});

xhr.open("POST", "https://graphql.prepr.io/<YOUR-ACCESS-TOKEN>");
xhr.setRequestHeader("Prepr-Customer-Id", "409ad1af-d644-4d3f-8cb9-691c0318a980");
xhr.setRequestHeader("Content-Type", "application/json");

xhr.send(data);

<?php
$client = new Client();
$headers = [
'Prepr-Customer-Id' => '409ad1af-d644-4d3f-8cb9-691c0318a980',
'Content-Type' => 'application/json'
];
$body = '{"query":"query{\\n  Page (id: \\"8da86294-c15d-415a-9fa3-6ea289c9403c\\") { \\n    _id\\n    title\\n    _slug\\n    stack { \\n        __typename\\n    ... on PageHeader {\\n        _id\\n        heading\\n        text\\n        cta_label\\n     _context {\\n                kind\\n                variant_id\\n            }\\n        }\\n    }\\n  }\\n}","variables":{}}';
$request = new Request('POST', 'https://graphql.prepr.io/<YOUR-ACCESS-TOKEN>', $headers, $body);
$res = $client->sendAsync($request)->wait();
echo $res->getBody();

Update the id string value of the query with the Content item ID of the Home page with A/B testing. See the image below on where to find the Content item ID on the content item page.

To retrieve the page content, call the API using the query above and pass a Prepr Customer ID as a header. The Prepr Customer ID is required to determine which variant a visitor gets. Get a customer Id as shown above and use this value to set the Prepr Customer ID header value. See an example in the below code snippet:

--header 'Prepr-Customer-Id: 409ad1af-d644-4d3f-8cb9-691c0318a980' 

As a response to your API request, Prepr sends the variant to be displayed. And your app then needs to build its response based on this variant. See an example of a response below:

{
  "data": {
    "Page": {
      "_id": "8da86294-c15d-415a-9fa3-6ea289c9403c",
      "title": "Home page",
      "_slug": "home-page",
      "stack": [
        {
          "__typename": "PageHeader",
          "heading": "Bring a friend to any baking course for free",
          "text": "Sign up for any baking course in our calendar together with a friend and they can join for free!",
          "cta_label": "Sign up now!",
          "_context": {
            "kind": "AB_TEST",
            "variant_id": "A"
            "group_id": "34022364-7ed3-46ff-9055-a208c5b0cccd"
          }
        }
      ]
    }
  }
}

The algorithm remembers which visitor gets which variant so that the same visitor always gets the same page header variant when interacting with your web app.

Add HTML attributes to the page header component

To track visitor interactions for A/B testing and get accurate reporting results, we need to add attributes to the page header. In our example, we have a Page header with some specific text and a call to action link for each variant, so we want to report on the following types of visitor interactions:

• Impressions: How many visitors looked at the page header for each variant.
• Clicks: How many visitors clicked the CTA link for each variant.

We can determine the click-through rate based on these two events, which will show which variant performed best. Because the above interactions need to be per variant, we also need a custom variable to identify the variant.

See an example code snippet to include the HTML attributes below:

<!-- Page header -->
<div
  id="{{__typename}}_{{group_id}}"
  prepr-variant="{{ variant_id }}"
  ...
>
<!-- Call to action link in the Page header-->
  <a
    href="#" 
    id="cta_link_{{__typename}}_{{group_id}}"
    prepr-variant="{{ variant_id }}"
    ...
  >Learn more</a>
</div>

In the above example, we set the Ids dynamically by using the following fields returned in the query response:

• __typename - The element type, for example, PageHeader.
• group_id - A unique Id to identify a specific A/B test group. This is useful to identify all elements that have an A/B test enabled.
• variant_id - This returns a value of A or B.

Using the query response above as an example the Ids get the following values:

• id = "PageHeader_34022364-7ed3-46ff-9055-a208c5b0cccd"
• prepr-variant = "A"

That’s it. You have integrated the A/B test into your website. A correct page header variant will be displayed every time a customer visits your website.

The next step is to set up the analytics to measure the click-through rate to see which variant performs best.

Step 2: Connect analytics to the front end

Prepr A/B testing works with any analytics tool. If you already have your analytics tool set up to measure visitor interactions on your web app, then you can ignore this step.

1. If not already done, work with the marketing team to connect GA4 and GTM to your web app by following the GA4 and GTM setup process.

Once completed, the analytics tool is connected to your front end. The rest of the analytics setup is ordinarily done by the marketing team and you can find more details in the Run A/B tests guide for marketers.

After some time the marketing team will be able to use the reports to determine the better performing variant and close the A/B test. This means that Prepr will deliver the chosen variant to your website visitors. The other variant will be removed from the Stack.

Congratulations, you have successfully set up A/B testing in your front end. Check out the next section if you are implementing a static website.

Implementing A/B testing for static (SSG) websites

Unlike dynamic sites, static sites serve pre-built content to a web browser without calls to a database. To perform A/B testing for static site rendering, you need to set up variants ahead of time (particularly before you build the web app) and then split traffic between different routes using the edge middleware (see an example tool). Let’s see it in detail.

The A/B testing for static site rendering looks like this:

1. On pages where an A/B test needs to be triggered, you call the API to pre-fetch variants. Your request must include the following fields:

• _context - system field that contains additional details about variants such as the following fields:
  • kind – returns a value of PERSONALIZATION or AB_TEST.
  • variant - returns a value of A or B if kind has a value of AB_TEST

Here’s an example code snippet. Update the id string of the query with your Content item ID (find the Id in the right-hand column on the Page content item in Prepr).

query{
    Page (id: "8da86294-c15d-415a-9fa3-6ea289c9403c") { 
        _id
        title
        _slug
        stack (personalize: false) { 
            __typename
        ... on PageHeader {
        _id
        heading
        text
        cta_label
                _context {
                    kind
                    variant_id
                }
            }
    }
    }
}

{
  "data": {
    "Page": {
      "_id": "8da86294-c15d-415a-9fa3-6ea289c9403c",
      "title": "Home page",
      "_slug": "home-page",
      "stack": [
        {
          "__typename": "PageHeader",
          "_id": "79e752ca-76e5-4b52-8bf9-4a5960d0ccee",
          "heading": "Bring a friend to any baking course for free",
          "text": "Sign up for any baking course in our calendar together with a friend and they can join for free!",
          "cta_label": "Sign up now!",
          "_context": {
            "kind": "AB_TEST",
            "variant_id": "A"
          }
        },
        {
          "__typename": "PageHeader",
          "_id": "695afdf3-6b3e-4c05-8e2e-39dae548969e",
          "heading": "20% off on all bread baking courses",
          "text": "Join our Spring promotion and save 20% on any baking course.",
          "cta_label": "Sign up today!",
          "_context": {
            "kind": "AB_TEST",
            "variant_id": "B"
          }
        }
      ]
    }
  }
}

2. When a user visits your web app, you make a one-time request to know if this user is assigned to Bucket A or Bucket B. Your request must include the following headers:

• Prepr-Customer-Id with a session Id of a visitor
• Prepr-Bucket-Customer = true

The bucketing runs at the edge location of the CDN and takes no longer than 7 ms. A response to your API request contains a header – either X-Prepr-Customer-Bucket A or X-Prepr-Customer-Bucket B.

See an example request below:

curl --location --globoff 'https://graphql.prepr.io/<YOUR-ACCESS-TOKEN>' \
--header 'Prepr-Customer-Id: 409ad1af-d644-4d3f-8cb9-691c0318a980' \
--header 'Prepr-Bucket-Customer: true'

3. With this information, the front-end middleware can redirect a visitor to a respective variant of your A/B test. For all the requests from the same visitor, the same variant will be shown.

Want to learn more?

Check out the following guides:

• Capturing event data
• Connecting your front-end application
• Personalization

Schedule a free consultation

Do you want to get started with A/B testing but still have questions or want a demo? Book a free appointment with a Prepr solution engineer right away.