All Collections
FAQs
How is this different from A/B testing?
How is this different from A/B testing?
Scott Gutelius avatar
Written by Scott Gutelius
Updated over a week ago

Some testing is better than none, but Motiva Message Testing AI goes way beyond A/B/n testing. Classic A/B/n tests send the same number of messages for each variation to learn what option is best. This means that as the number of message variations grows, you are sending more customers an inferior message during the testing period. 

In contrast, Motiva Message Testing AI experiments with any number of variations over time, incrementally learning which work best, and adapting to maximize overall performance. Instead of static batch 'n blast, you create an email step that listens and adapts.

For this reason, think beyond testing just two versions. Why?

If you give Motiva two versions, it will run it similarly to a traditional A/B seeking to get to a single winner according to the Confidence Threshold that you set. This forces Motiva to send the same volume of each of the variations out until it finds that winner. The downside here is that Motiva isn't incrementally adjusting its send strategy along the way, since that would pollute the statistical validity of the results.

If you are running only two variations, you'll likely want to lower the Confidence Threshold to 85% or 90%. This will force Motiva AI to try to find a winner sooner.

There's nothing wrong with only two variations or with A/B tests. It's just not taking fuller advantage of what Motiva can do for you, and your team isn't getting as much feedback or learning along the way.

Did this answer your question?