Skip to main content

AB testing

Louise
  • Updated
websitepersonlisation_tag@2x.png
Note: This article is about AB testing for Ometria's website personalisation feature. For AB testing your email, SMS and Push campaigns, see Split tests in automation campaigns

Use AB testing to compare your different merchandising strategy variables and see which performs better.

Set up an AB test

Go to: Website personalisation > Merchandising strategies > select a Page Element from the list. 

Select Create a new strategy and choose AB Test:

See also: Merchandising strategies

Next, toggle on the variables you want to test:

Note: We recommend choosing one variable per AB test. This is to make sure that the test is focussed and the maximum amount of data is available.

Click Next and complete the fields:

Field Description
A/B Test Name Give your A/B test a memorable name.
Variable This field auto-completes based on the variable you selected in the previous step. 
Objective Define the success metric. Choose from:
  • Views
  • Purchases
  • Adds to basket
Success threshold

Use the slider to define the threshold for your success metric, e.g. 95%

You might want to reduce this for lower traffic pages. 

In the table at the bottom of the screen, select your variables (e.g. the blends you want to test):

Now select the Strategy tab at the top of the screen:

Complete the fields here:

Tip: If you're creating a simple AB test, you could leave the majority of the fields blank to inherit the default strategy for your page element. The only change you need to make is to the variable you're testing, e.g. the blend.

Field Description
Disabled/Enabled
  • Disabled - This strategy is not live and active on your site.
  • Enabled - This strategy is live and active on your site.

We recommend leaving your strategy disabled until you've completed your setup. 

If you save this merchandising strategy without setting a Schedule, and have set the strategy to Enabled, it runs immediately and executes if the conditions are met.
Name

Enter a memorable name for your new strategy.

This can be the same as the name for your AB test. 
Description Enter a clear description.
Products per element

Choose how many products should display in this page element.

If you leave this blank, this strategy will inherit the number from your default strategy, or from previous strategies on the page element. 
Merchandising blends

Select a merchandising blend. 

If you leave this blank, this strategy will inherit the number from your default strategy, or from previous strategies on the page element. 
Merchandising templates

Select a merchandising template.

If you leave this blank, this strategy will inherit the number from your default strategy, or from previous strategies on the page element.
AdSpots

Select an AdSpot.

If you leave this blank, this strategy will inherit the number from your default strategy, or from previous strategies on the page element. 
Group by

Select a product grouping.

If you leave this blank, this strategy will inherit the number from your default strategy, or from previous strategies on the page element.
Templates

Select a template.

If you leave this blank, this strategy will inherit the number from your

default strategy, or from previous strategies on the page element.

Save your changes.

If you've selected Enabled, then your AB test is up and running as soon as you save it.

Note: It takes about an hour for data to start flowing in.

AB test results

Return to your AB test at any time and select the test tab :

It typically takes one hour to start seeing data after your AB test has been enabled:

When a visitor lands on a page that triggers the AB Test strategy (as defined by the triggers you set) there’s a 50% chance they'll see option A or B. 

(If you had three variables in your test, e.g. A, B and C, it would be a 33% chance, and if you had four; A, B, C and D it would be a 25% chance, etc.)

Visitors' actions are tracked against the variable they saw:

  • Impressions - The number of times a product has displayed on a page element
  • Views - The number of visitors who have clicked through to view a product page
  • Conversation rate
  • Probability to be best

AB test winners

When a variable has won the test, it is promoted and used as the active variable (blend, template, AdSpot, etc.) in your strategy, which continues to run as normal.

Once that happens you can make any changes you like to the strategy, or disable it if you prefer. 

Manually declare a winner 

You can also manually declare a winner.

Open your AB test and select Options > Finish A/B test:

You're given the option to either Abandon the test (disabling it) or selecting a winner. 

Make your choice and select Apply. 

Related articles

Comments

0 comments

Article is closed for comments.