Split tests - A/B testing

In Ometria, you can create split tests (also known as A/B tests) for automation and broadcast campaigns.

• Split tests in automation campaigns
• Split tests for broadcast
  • Split test with winner

How splitting works in Ometria

It’s important to understand that Ometria does not ‘split your contacts down the middle’, and you should not expect a totally equal split of your contact numbers.

Splitting contacts in Ometria is random for each contact entering the campaign.

So, if you have a 50/50 split test node, any contact that enters the campaign has a 50% chance of entering either split - it’s the same as tossing a coin for each contact.

This method keeps your split tests dynamic, meaning that you don’t need to go back and create a new split when new contacts enter the flow.

The importance of sample sizes

Because the split (or ‘coin toss’) happens for each contact, there are lots of different possibilities for your split sizes. This is why it’s best to have a large group of contacts when you intend to perform a split test.

For example, if your sample size is only 20 contacts and you create a 50/50 split, you might end up with any of the following combinations (and more):

Whereas with a sample size of 1000 contacts, a 50/50 split could result in any of the following (and many more)