Run Your Best Sticky Call To Action Experiment

With 194,400 possible variable combinations across 29 experiments, past lifts have ranged:

High power estimate +1.5%
Highest estimate +9.1%
72% positive · 29 tests
Taught
by
Jakub
Linowski

Scope

Choose the page type where the test runs.

Prototype

PHOTO

Specs

Reviews

Treatment Variables

Set each option, then check the variables your B version changes.

•
•
•
1 test

Experiment Design

Choose the structure of your experiment.

Targeting / Triggering

Refine where and when the test fires, beyond the page types chosen above in order to increase power (and shorten the testing duration).

Metrics & Ship Requirements

Mark each metric as Primary, Secondary, or Guardrail, then set the requirement it must meet to ship.

Metric Primary Secondary Guardrail Superiority Non-Inferiority Binary Continuous
%
%
%
%
%
%
%
%
%
%
%
%
%

Power, Sample & Duration

The effect you want to detect and how long you'll run to reach the power for it.

%
%
%
%
Total Sample — visitors across all variants
Duration —weeks  

Meta-Analysis

Combine your experiment with that of others to reach a decision faster or increase power.

Start Rules

When can the experiment start?

Stop Criteria

When to end the experiment. Pick both for a hybrid.

Stop at

Segments

Which subgroups should we track and break the results down by, to see if the effect differs across them?

Segment Exclude Measure
Long: Benefits most from a persistent sticky CTA
Many (5+): More choices may increase value of a sticky anchor

Iteration Rules

What next experiments to run once the results come in.

Outcome → Re-run with: Higher intensity Higher power Elsewhere Something different Invert Move on
IF Stat sig primary A clear, significant win.
IF Stat sig primary Flat secondary A win on primary, no effect on secondary.
IF Flat primary Stat sig secondary No effect on primary, a win on secondary.
IF Flat primary Flat secondary No significant effect on either.
IF Stat sig neg primary A clear, significant loss.

Export to Experimentation Tools

Pick a destination — copy a plain-text spec, or a JSON config for your platform.


        
        
    

Now Run This Experiment