Early On
Early on while Motiva AI is still learning, the results screen will be pretty sparse. You may even encounter this message:
This means that the campaign is still awaiting reports from Eloqua, or the number of sends for each email option has not reached the 150 minimum. (For instance, if you are testing 3 variations, the minimum number of contacts necessary would be 450.)
Once the 150 sends per email threshold is reached and results are available, the number of sends per variation will be uniformly distributed between the email options, as seen in the screenshot below:
At this point, Motiva AI doesn't know anything about which variations are better than the others.
Screen Layout
The screen has the following areas from top to bottom:
Message -- Motiva tells you what's going on.
Date Progress -- for Bulk emails only. (Drip emails don't have a set end date.)
Details Table -- Early on, this table shows all message variations, but as Motiva AI learns, it will only show Top Performers.
Rates Chart -- Open or Clickthrough Rate per variation over time
Daily Sends Chart -- Sends per day per variation
We Found a Winner!
Once Motiva AI starts to learn, more information will be shown in the dashboard. In the following example, the email was sent for 3 days, and Motiva AI found a single winner -- see the star next to the "Webinar Reminder A" email.
The screen has been automatically filtered to show only the top performer. The filtering is done based on the Confidence value seen in the table. Only the top N email options whose sum of their confidence is greater than or equal to the Confidence Threshold will be displayed by default.
You can toggle the screen to show ALL email options using the button in the upper right. Here's the same screen but showing ALL email options:
In the example above, Motiva AI found a single message variation whose confidence exceeded the confidence threshold on the very first day. After that, Motiva AI did not spend anymore time playing the other options.
But many times, Motiva AI will not find a single winner. It will take a few message variations for the confidence to sum to the confidence threshold. In this case, the message area will say something like, "With high confidence, Motiva AI found the Top 2 Performing Emails." Here's an example of that:
Another thing to remember is that the rates used by Motiva AI's models are estimates of the actual rates -- either open or clickthrough. And those rate estimates have uncertainty, which is represented by the shaded area around the lines in the rate chart.
Generally, as Motiva AI's models process more data (i.e. contacts), that uncertainty diminishes, and we become more confident in our rate estimates.
You can learn more about how Motiva AI handles uncertainty here:
That's it! If you have any questions, click the little green circle in the lower right corner and give us a shout. We're happy to help.