The PFA enables marketers to fully understand how their campaigns are performing. Users can analyse the path that each recipient took, how they progressed and how they interacted with their emails.
This is available in the “individual campaign performance” dashboard which is accessible by clicking on any automated campaign in the “automation campaigns” dashboard and “marketing dashboard”:
This dashboard solves several use cases. Users are able to analyse:
- how many contacts have passed through each individual node in test and live mode
- revenue performance per segment, split test and condition node
- email performance per node as well as per segment, split test and condition node
Below are some use cases that many of our customers can benefit from (the screenshots below are from our demo account and in no way have enough data to represent statistical significance, this is purely for demonstration purposes):
Split test performance in terms of revenue and email stats such as open rate and click to open rate:
The screenshot below gives you a snapshot of one of the segment paths of a multi stage, segmented campaign. By hovering over the split test node (left screenshot), you can analyse total revenue for each split path and email stats such as emails sent, revenue per email, average open rate and average click to open rate.
The screenshot below gives the user a very clear indication that the first split test is performing better based on open rate and the click-through-rate.
1. Analyse which flows generate sufficient send volumes, to decide whether you need to allocate resources to create additional templates:
With the campaign builder, you can easily segment any campaign into any path you want. For example, when setting up an advanced browse abandonment campaign for one client, we segmented the campaign into top performing categories such as t-shirts and sweaters and then split those into women / men. After running the campaign for a few days with a dummy template we were able to see how many contacts had passed through each path and node.
Based on the data we were able to identify which paths we should focus on and allocate resources for template creation.
2. Analyse the difference in behaviour between your VIP, active, at risk and lapsed customers (or any other segment):
The biggest bottleneck for any retailer of any size is template creation. However, there are many use cases that can be solved without any additional creative work. For example, every single campaign can be segmented into VIP, active, at risk and lapsed customers in 5 minutes. With enough data, marketers can start recording customer behaviour and revenue differentials between each segment, without the need to create any additional templates.
3. Other use cases
There are a few other use cases that can help a retailer leverage the campaign builder and PFA to better understand customer behaviour. For example, you can create a A/B/C/D test to evaluate performance of different recommendation engines. There is no need to modify the template, all you have to do is create a recommendation instance in the new product recommendation UI and update it in the action node.
You can also make it even more advanced by segmenting paths into top performing categories and filtering product recommendations based on that category. Once again, there is no need to modify or create new templates.
Other use cases that require minimal work:
- split test subject line
- test subject line per segment / category
- split test copy
- test copy per segment / category
Once enough data is collected the user can use this information in the PFA to optimise their campaigns to improve customer experience and generate more revenue.
To give you a better understanding, a fully personalised browse abandonment campaign, with product recommendations and segmented header images for 7 different paths (7 templates) took about three hours to create.
Important: It’s very important to note that there are discrepancies between the overview dashboard and PFA dashboard for historic campaigns that have been started before this feature was released. In order to achieve the most accurate results it’s best to restart a campaign that was created before the release date.
However, even if you start / restart a campaign after the release of PFA there might still be discrepancies for the following reasons:
Discrepancies not due to lack of recording data for historical campaigns
(1) There can be a discrepancy in campaign revenue versus node revenue when someone forwarded an email to someone else who then made a purchase (they are part of campaign revenue but not node revenue).
(2) The coupon code in an email could have been given to someone else (not necessarily by forwarding the email). This can lead to an order being attributed to the campaign but that person may not have been part of the campaign.
(3) The retailer may have stopped / started the campaign and then someone from the previous campaign run goes on to make a purchase during the current campaign run. In the UI (where you're only filtering on timestamps) you'd see this as revenue of the current campaign run (but not as part of a node's revenue).
Discrepancies due to technical reasons
(4) The accuracy of numbers shown in the path flow analysis can be affected by the data that we have for a campaign. Over time Ometria has been tracking more information for triggered campaigns which means that older campaigns have less information available for reporting.
This means that historical information was missing for some campaigns which had to be calculated manually.
Examples of this include:
- We have a history of emails sent for a campaign but this originally did not include what email node that email came from. As of June 2015 we have been tracking what particular node in a campaign sent a particular email.
- We have information regarding people that are part of a campaign but this originally did not include the exact path that someone took through the campaign. As of June 2015 we have been tracking the exact path that someone took through a campaign.
- In November 2015 we introduced a notion of what makes an entry into a campaign "count" for the purposes of reporting. For example, someone who enters a campaign and drops out immediately after a segment node doesn't "count" as neither the contact nor the retailer can see an effect of that campaign on that contact. This information had to be calculated for entries prior to November 2015 to ensure that numbers match.
The reported numbers can also be affected when changes were made to the campaign structure at some point while the campaign was running. The best results can be accomplished by stopping and starting the campaign to ensure that we have the most accurate information available for every contact that goes through the campaign.
Important: when comparing stats on the overview tab and the PFA tab, make sure that the timeframe that you are looking at matches the start date that can be found under the campaign name on the overview tab.
Description of how the reported node statistics are calculated
The first thing to note is that we only use entries into the campaign after the campaign started. If the campaign is in test mode then that only means test entries. If the campaign is live then that means only live entries.
The second thing to note is that the system does not display, say, revenue per email in case there are no email nodes in that flow (no email nodes = no revenue per email = no value displayed).
Number of pass throughs
The number of contacts that went through that node and that also went through at least one "actioned" node (for example, an email node). An actioned node is a node that we consider to make that entry into the campaign "count". For example, if you enter into the campaign and immediately drop out after, say, a segment node then that entry doesn't count.
The number of emails sent from that email node. As mentioned above this is the number of test emails when the campaign is in test mode and the number of live emails when the campaign is live.
For a segment node this is the total number of emails sent in the flow(s) that follow the segment node.
The number of emails that were sent from this node and that were opened at least once.
The number of emails that were sent from this node and that were clicked at least once.
The number of emails opened divided by the number of emails sent or zero in case emails sent is zero.
Click through rate
The number of emails clicked divided by the number of email sent or zero in case emails sent is zero.
Click to open rate
The number of emails clicked divided by the number of emails opened or zero in case emails opened is zero.
For a segment node this is the sum of the revenues generated after each of the split nodes.
For a split node we calculate the following:
- Find all valid transactions attributed to this campaign (either due to in-session or coupon code) that were placed after the campaign started. We don't look at the time between an email and a transaction here, we only look at transactions that are attributed to the campaign regardless of how long after an email they were placed.
- For the customers of those transactions, look at all their campaign entries "that count" (as explained earlier) after the campaign started and find those that have always performed the same path through the campaign.
- For those customers, sum the revenue of the aforementioned transactions and attribute it to this split node.
Revenue per email
For a segment node this is the sum of revenue of each of the split nodes divided by the total number of emails sent after each of the split nodes.
For a split node this is the sum of revenue generated in the flow that follows it divided by the number of emails sent in the flow that follows it.
If no emails are sent then revenue per email has no value and it will not be displayed in the UI for the segment or split node.
Average open rate
For a segment / split node this is the total number of emails opened in any of the nodes that follow it divided by the total number of emails sent in any of the nodes that follow it.
This value only shows up if one or more emails were sent.
Average click to open rate
For a segment / split node this is the total number of emails clicked in any of the nodes that follow it divided by the total number of emails opened in any of the nodes that follow it.
This value only shows up if one or more emails were opened.