How does it work?

How does it work?

Katrine Andersen & Ewa Seiler - October 27, 2017

In order to test which versions of the elements listed above work best for your specific target group, you need to approach the test in the right way.

First you will need to define the element that you want to test and generate the material you need for the test. If you decide to test a subject line, for example, you should prepare the two different subject lines to be used in the newsletter.

The next step is to divide your newsletter list into three groups: test group A, test group B, and the winner group.

 

How does A/B testing work

 

Typically, test group A will consist of 25% of the subscribers on your newsletter list, test group B will be another 25% of your list, and the third, the biggest group, will consist of the remaining 50% of your list. If you newsletter list is big enough, it might also be sufficient to divide your list into 10%, 10%, and 80% splits. Most email marketing systems will have this functionality included in the system, so you won’t have to worry about manually creating these splits. But keep in mind that in order to get statistically relevant results you will need to at least 50 subscribers to perform the desired action (e.g. opening the email). If this is not the case you risk your results not being a valid indicator of what works best.

Once you have defined the split of the test you will need to select how long the test should last, as well as how the winning version will be determined.

When your newsletter is launched, test group A will receive the A variation of your newsletter and test group B will receive the B variation. After the predetermined period of time allotted for the test, the winning version will be selected and sent out to the remaining profiles in your target group.

When using A/B testing in automated trigger flows, we recommend that you implement a 50/50 split for the two versions of the email you are testing. You will typically have to manually select a winner (by simply removing the losing version from the flow), as the timing of the test depends a lot on the volume of subscribers passing through your trigger flow.