# How do I A/B test a subject line or email in a campaign?

> On an email step, click Test another version of this email. Warmerly sends up to four versions in rotation and picks the winner once each has enough sends.

Source: https://warmerly.com/docs/help/ab-test-email-copy

On the campaign's **Sequence** tab, open an email step and click **Test another version of this email**. You then have version A and version B side by side, and you can add up to four versions in total with **Add version C** and **Add version D**. Warmerly sends the versions in rotation, and once every version has had enough sends it picks a winner and uses only that one from then on.

## Steps

1. Open the campaign's **Sequence** tab and expand the email step you want to test.
2. Click **Test another version of this email**.
3. Edit each version's **subject** and **body**. Change one thing between versions (the subject, or one part of the body) so you can tell what made the difference.
4. Set **Decide after N sends each**. Every version must reach this number of sends before a winner can be chosen. Larger is more reliable and slower.
5. Launch, or keep the campaign running. Progress is shown under the versions.

## How versions are handed out

Each new lead gets whichever version has been sent the least so far, so the split stays close to even across however many versions you have.

While a test is running, the versions are the step's copy. What you typed in the step before enabling the test becomes version A. If you turn the test off, the winner (or version A if there is none yet) goes back into the step.

## How the winner is picked

- Every version must have reached the **Decide after** number of sends.
- The last email of each version must be at least 24 hours old, so recipients have had time to answer.
- **Replies decide it.** As soon as any version has a reply, the highest reply rate wins. Only genuine replies count, not out-of-office auto-replies.
- **Opens are the fallback.** If no version has a reply yet but some have opens, the highest open rate wins. Opens are only recorded when **Track opens** is on for the campaign, and they are unreliable, which is why replies come first. See [Open and click tracking](https://warmerly.com/docs/help/open-and-click-tracking).
- If there are no replies and no opens at all, nothing is picked. Warmerly will not crown a winner with no evidence.

When a winner is chosen, the step shows "Version X won" and every later send uses it. The losing versions can no longer be edited. Results per version (**Reply rate**, **Open rate** and a **Winner** badge) appear on the campaign's **Overview** tab under **Variant performance**.

## What it cannot do

- The winner is simply the highest rate. It is not a statistical significance test, so a small sample can pick a version that is not really better. Use a sample size big enough to trust: a rate off a handful of sends is noise.
- If you change both the subject and the body between versions, a winner tells you which email did better, not which change did it. The editor warns you about this, and warns if two versions are identical.

## Common problems

**The test never picks a winner.** Check that every version has reached the sample size, that at least one reply exists (or that open tracking is on and at least one open exists), and that 24 hours have passed since the last send of each version.

**Open tracking is off, so the test is slow.** Then replies decide it. That is the better signal, it just needs more sends to settle.

**I need the copy in one place.** After a winner is chosen, turn the test off and the winner goes back to being the step's own copy.

## Related

- [How do I write an email sequence?](https://warmerly.com/docs/help/email-sequences)
- [What stops a campaign sequence for a lead?](https://warmerly.com/docs/help/what-stops-a-sequence)
- [What the Warmi advice on a campaign means](https://warmerly.com/docs/help/campaign-advice)

## More in Campaigns & inbox

- [How do I personalise every email with the AI opener?](https://warmerly.com/docs/help/ai-opener): Put the aiopener tag at the top of your email. Warmerly writes a one or two sentence opening line per lead…
- [How do I see who has been contacted in a campaign?](https://warmerly.com/docs/help/campaign-audience): A campaign's Audience tab lists every lead with its status and how many emails have gone out. Here is how to…
- [What do the campaign statuses mean (draft, sending, paused, completed)?](https://warmerly.com/docs/help/campaign-statuses): A campaign is a draft until you start it, then sending, scheduled or needs attention while live. What each…