Most popup A/B testing dies on button colors. Coral versus teal, "Subscribe" versus "Sign up," a rounded corner versus a sharp one. You run it for two weeks, the difference is a rounding error, and you conclude testing doesn't work. The problem wasn't testing. It was testing the smallest possible variable. Here are six experiments that actually move popup conversion, ordered by how much payoff you should expect — biggest first.
For each, you get the hypothesis, how to run it as an A/B variant in WooHoo (split-testing variants is a built-in feature, so none of this needs code), and what usually wins directionally. Directionally — because the exact number is specific to your store, and anyone who quotes you a universal percentage is selling something.
1. Trigger timing
Hypothesis: when a popup fires matters more than what it says. A behaviour-based trigger beats a blunt timer.
How to run it: build two identical popups. Variant A fires on a time delay (say, five seconds). Variant B fires on a behaviour signal — scroll depth past a product, exit intent, or a second pageview. Split traffic evenly and read conversion by variant.
What usually wins: the behaviour trigger, and often by a wide margin. A shopper who's shown intent is a different prospect from one who just landed. This is the highest-leverage test on the list, which is why it's first.
2. Offer type: percent vs dollar vs shipping
Hypothesis: the structure of the incentive outperforms its size. "15% off" and "$15 off" and "free shipping" pull differently even at similar values.
How to run it: hold the design constant and vary only the offer across three variants. Percent off, a dollar amount, and free shipping. Watch conversion and, if you can, redemption and resulting AOV — not just opt-in.
What usually wins: it depends on price point. Free shipping tends to punch above its weight on lower-AOV stores; a dollar amount reads as more concrete on higher-priced carts. The lesson is to test the shape of the offer before you ever argue about its size.
Measure past the opt-in.
An offer that wins on signups but loses on redemption or margin isn't a winner. Trace each variant to revenue where you can. A popup optimising for the wrong metric optimises confidently in the wrong direction — see popup metrics that actually matter.
3. Gamified vs static
Hypothesis: a game beats a form on the same offer, because it lowers the cost of the first action.
How to run it: variant A is your current static email box. Variant B is a spin-to-win popup for Shopify or scratch card offering the identical discount. Same offer, same trigger, same audience — the only difference is the mechanic.
What usually wins: the game, consistently, because "want to play?" is an easier yes than "give us your email." This is the test that tends to produce the biggest single step-change if you're still running a static form — the mechanics are unpacked in our anatomy of a 13.2% popup.
4. Headline frame
Hypothesis: leading with the payoff beats leading with the mechanism.
How to run it: two variants, same offer. A describes the action — "Join our newsletter." B leads with the reward — "Win up to 40% off your first order." Everything else identical.
What usually wins: the payoff-first frame. Shoppers decide whether to engage before they finish the sentence, so the benefit has to come first. This is a cheaper test than the ones above, but it's real — unlike button color.
5. Opt-out copy
Hypothesis: the "No thanks" line lifts opt-ins when it names the loss honestly.
How to run it: keep the whole popup fixed and vary only the opt-out. A is the default "No thanks" or "Close." B is a first-person loss frame — "No thanks, I'll pay full price."
What usually wins: the loss frame, gently. It's a small lever, but it's free, and it stacks on everything above. Just don't cross into confirm-shaming — the full argument is in the "no thanks" link is half your popup.
6. Mobile-separate layout
Hypothesis: a popup designed separately for mobile beats one that's merely responsive.
How to run it: against your current responsive popup, test a mobile variant built for thumbs — larger touch targets, the input positioned so the keyboard doesn't cover the action, a game sized to feel real on a small screen.
What usually wins: the purpose-built mobile layout, because most of your impressions are phones and "shrunk desktop" quietly fails there. More in mobile popups: three rules.
Testing button colors is how stores feel busy without learning anything. Test the trigger, the offer structure, and the mechanic — that's where the movement is.
Split-test any of these without code.
WooHoo runs A/B variants natively — clone a popup, change one thing, split the traffic, and read conversion by variant. Start with the trigger test.
Run them in order
Work the list top-down, because it's ordered by expected payoff. Fix your trigger first, then test the offer structure, then the mechanic, then the framing, then the opt-out, then mobile. Change one variable per test, give it enough traffic to mean something, and keep only what beats the control. Six real tests will teach you more in a quarter than a year of button colors ever will.
Frequently asked questions
What should I A/B test on a popup first?
Trigger timing. When a popup fires — on a behaviour signal like scroll depth or exit intent versus a blunt timer — moves conversion more than almost any other variable. Test that before you touch copy, color, or design, because a well-timed average popup beats a badly-timed perfect one.
How long should a popup A/B test run?
Long enough to reach a meaningful sample at your traffic level — typically two weeks or several hundred conversions per variant, whichever comes later. Change only one variable per test so you can attribute the difference, and trace winners to redemption and revenue, not just opt-in rate.
Does WooHoo support A/B testing?
Yes, natively. You can clone a popup, change a single element, split traffic between the variants, and read conversion by variant from the dashboard — no code required. That makes it practical to run the six tests in this piece one after another.
// Keep reading
Head of Growth Research at WooHoo. A decade of lifecycle and CRO work across DTC brands. Plays bass.
