Running A/B tests with Beam

Conducting experiments on users to optimize on signups or other metrics is a really common tactic for online businesses. With Beam, you can use custom events to gather the data you’ll need to make comparisons.

Let’s use an example to show how this works - which landing page gets users to click the ‘Get started’?

Basic Setup

You have two landing pages, both of which have a ‘Get started’ button which takes the user to the sign up page. The control and treatment pages might have different copy, colors or images.

controlLandingPage.html treatmentLandingPage.html

Below, we’ll walk through the steps on how to use custom events to log the data, and then how to use the funnels to display the data.

Step 1: Randomize users to get one of the landing pages

So we want to randomly assign each new user to one of the landing pages. For each new user, you could randomly generate a number between 0 and 1 and call it assigning_num. So the code could be something like:

If assigning_num < 0.5: Give user controlLandingPage.html Else: Give user treatmentLandingpage.html

This step doesn’t require the use of custom events.

Step 2: Create a custom event for the randomization

When a user is assigned one of the landing pages, we’ll want to define a custom event so we know how many were assigned to each arm.

So the code might look like:

If user is assigned controlLandingPage.html:
	  window.beam(‘/custom_events/controlLandingPage’)
Else:
	pass

So this simple if / else statement just says that if the user hits controlLandingPage, then log some data for the page ‘custom_events/controlLandingPage’.

Create a similar custom event but for the treatmentLandingPage.

Step 3: Create a custom event for clicks on the ‘Get Started’ Button

We also care about the number of users that click on the get started button. So again, we create custom events.

If user clicks on ‘Get Started’ while on controlLandingPage:
	 window.beam(‘/custom_events/button_click_control’)

So this just says that if a user, who has been assigned to the control landing page, clicks on the relevant button, send that data to Beam’s server.

And again, we do the same thing for the treatment landing page.

How do I see my data?

Go into the funnel analysis tab and enter the two steps as follows.

So you’ll now see the click through rate that the control landing page gets. In a separate tab, repeat this but for the treatment page. And now you have the two numbers you want from your A/B test.

Custom events are very powerful. You can define when you want to log some data and create a custom path for it. And when combined with the above workflow, you can easily create A/B tests.

We’re working on improving the workflow and polish for running A/B tests with Beam. But hopefully this gives you a sense of what you can do. If you have any questions, please let us know at hi@beamanalytics.io.