A/B Test

Netflix Style Pricing Pages

Netflix A/B tested at least three pricing plan layouts between 2019 and 2026. Now, in 2026, a third version is live.

QuestionIs Netflix' latest pricing plan layout really the best, or can you help discover something they don't know?

MDE range 1-3%
With
defaults
from
Ishan Goel Founder of Thinking Bell, ex-VWO Stats Jakub Linowski Founder of GoodUI

Run this experiment for: ImpactNoiseVelocityExplorationIterationQuality

Your existing pricing page

Choose your plan

Downgrade or upgrade at any time.

Monthly price
$9.99
$13.99
$16.99
HD available
Ultra HD available
Screens you can watch at the same time
1
2
4
Watch on your laptop, TV, phone and tablet
Unlimited movies and TV shows
Cancel anytime
HD and Ultra HD availability subject to your Internet service and device capabilities. Not all content available in HD or Ultra HD.

Choose your plan

Downgrade or upgrade at any time.

Choose your plan

Downgrade or upgrade at any time.

Monthly price $16.99
Resolution Ultra HD
Screens 4
Laptop, TV, phone & tablet Yes
Unlimited movies & TV shows Yes
Cancel anytime Yes
Ads No ads

Treatment Variables

To increase your odds of success, maximize impact and decrease sample size. Impact

Treatment variableInclude in B C D E Future
Evidence ↑ 1 of 1 test · +10.2% (Test 525)
Evidence ↑ 1 of 1 test · +9.3% (Test 319)

Targeting / Triggering

Refine where and when the test fires, beyond the page types chosen above in order to increase power (and shorten the testing duration). NoiseVelocity

Variance Reduction

Cancel out noise you can predict, to get more power from the same traffic. NoiseVelocity

th percentile

Set the cap before the test starts: capping slightly changes what is measured.

Metrics & Ship Requirements

Mark each metric as Primary, Secondary, or Guardrail, then set the requirement it must meet to ship. ImpactIteration

Metric Primary Secondary Guardrail Superiority Non-Inferiority Binary Continuous
%
%
%
%
%
%
%
%
%
%
%

Power, Sample & Duration

The effect you want to detect and how long you'll run to reach the power for it. Noise

%
%
%
%

Total Sample — visitors across all variants
Duration —weeks  

Meta-Analysis

Combine your experiment with that of others to reach a decision faster or increase power. NoiseVelocity

Start Rules

When can the experiment start?

Stop Criteria

When to end the experiment. Each early stop trades some power for a shorter run. Quality

Stop at

Segments

Which subgroups should we track and break the results down by, to see if the effect differs across them? Iteration

Segment Your Bet Exclude Measure

Iteration Rules

What next experiments to run once the results come in.

Outcome Re-run with: Higher intensity Higher power Elsewhere Something different Invert Move on
IF Stat sig primary A clear, significant win.
IF Stat sig primary Flat secondary A win on primary, no effect on secondary.
IF Flat primary Stat sig secondary No effect on primary, a win on secondary.
IF Flat primary Flat secondary No significant effect on either.
IF Stat sig neg primary A clear, significant loss.

Now Run This Experiment