Early in the Campaign
In the early part of a campaign, while Motiva AI is still learning, the results screen will be pretty sparse, and the number of sends per variation will be uniformly distributed between the email options, as seen in the screenshot below:
At this point, Motiva AI doesn't know anything about which variations are better than the others.
The screen has the following areas from top to bottom:
- Message -- Motiva tells you what's going on.
- Date Progress -- for Bulk campaigns only. (Drip campaigns don't have a set end date.)
- Details Table -- In the early part of a campaign, this table shows all message variations, but as Motiva AI learns, it will show only the Top Performers.
- Rates Chart -- Open or Clickthrough Rate per variation over time
- Daily Sends Chart -- Sends per day per variation
We Found a Winner!
Once a campaign gets rolling and Motiva AI starts to learn, more information will be shown in the dashboard. In the following example, the campaign ran for 3 days, and Motiva AI found a single winner -- see the star next to the "Webinar Reminder A" email.
The screen has been automatically filtered to show only the top performer. The filtering is done based on the Confidence value seen in the table. Only the top N email options whose sum of their confidence is greater than or equal to the Confidence Threshold will be displayed by default.
You can toggle the screen to show ALL email options using the button in the upper right. Here's the same campaign but showing ALL email options:
In the example above, Motiva AI found a single message variation whose confidence exceeded the confidence threshold on the very first day. After that, Motiva AI did not spend anymore time playing the other options.
But many times, Motiva AI will not find a single winner. It will take a few message variations for the confidence to sum to the confidence threshold. In this case, the message area will say something like, "With high confidence, Motiva AI found the Top 2 Performing Emails." Here's an example of that:
Another thing to remember is that the rates used by Motiva AI's models are estimates of the actual rates -- either open or clickthrough. And those rate estimates have uncertainty, which is represented by the shaded area around the lines in the rate chart.
Generally, as Motiva AI's models process more data (i.e. contacts), that uncertainty diminishes, and we become more confident in our rate estimates.
You can learn more about how Motiva AI handles uncertainty here:
That's it! If you have any questions, click the little green circle in the lower right corner and give us a shout. We're happy to help.