Marketing Tools – Split Tests
Split Tests allow you to test a message by sending versions out to a proportion of your Audience.
You can then select a winner manually or automatically (set in your Email Settings).
For example, you may want to test a subject line, or image, to see if more supporters open the email or take action.
You cannot currently have a Split Test for a message that contains conditional blocks.
Creating a split test
You can create a split test once you have created your first message. To do this, go to the Message stage and hover over the message you want to split test.
Click the “Split Test” icon and you’ll be shown a list of split test versions of that message. If you haven’t yet created any, you’ll just see the Core message or Audience Segment message if using a conditional email.
To add a new split test, click “Add Split” and then choose to create a message from a template (“Start from template”), or a version of an existing message, e.g. your Core message (“Clone Existing”).
Start from template
If you choose to start from a template, you will load the email builder as if starting from scratch.
Clone Existing
If you choose to “Clone Existing” then you choose which message to clone.
The “Name” of the message shows what version of the message it is. If it ends with “- Core” it is your Core message, whereas Audience Segments (conditional whole emails) are suffixed with the name of the profile, e.g. “- Donors”.
If you have already created a split test, then the name of the email will show the profile name and “[Split: B]” for the second version of that message, e.g. ” – Core [Split: B]” or ” – Donors [Split: B]”.
Select the message and click “Select Message” to begin editing it.
Editing a split test
The split test will be an identical copy if you clone, including any conditional blocks.
Simply edit the message as usual to make the changes you want to test on. It is recommend you only test on one thing so you can determine what changes have been successful.
For example, edit the Subject line.
Click Save to save your split test version.
If you want to edit the Split Test after you have closed it, then hover over the Message in the Message stage and click “Manage Split Tests” and then click the email icon in the message list.
Deleting a split test
To delete a split tests, hover over the Message in the Message stage and click the delete icon next to the version to delete.
Testing a split test
The Testing stage shows you the different versions of your message. Split Tests show as “A/B” next to the list if the message has a split test enabled.
To test the message, select “Send all variations of conditional content” and the different versions will be emailed to the test addresses when you click “Send Test Messages”.
Sending a split test
Split Test % - Percentage of the audience who will receive the test emails. For example, if the campaign has two messages and you choose 10% of the audience, 5% of the audience will receive one email and another 5% will receive the other email. Then after a winner is selected, the remaining 90% of the audience will be sent the winning email.
Split test winner - This is where you determine how the winner will be chosen for the split test. The winner will chosen option requires a winner be manually selected by a User. Automatically select a winner allows the User to choose the criteria that will determine the winner and the number of hours before a winner is selected. If the split test account setting is enabled, the automatic option will be the default selected and the settings will match the account settings (but can be changed).
If you have set it to be manually sent, then you can go to the Reporting stage which is accessed via the list of broadcasts.
After reviewing the reporting, you can then choose the split test version, and click Select winner and send and the remaining percentage of supporters not included in the test will receive the selected version.