Onboarding A/B Tests
Run structured A/B tests on onboarding surfaces — product tours, welcome emails, empty states, setup wizards — to systematically increase activation rate. Progresses from a single manual experiment to an autonomous AI loop that monitors onboarding health, generates hypotheses, runs tests, and implements winners.
npx gtm-skills add product/onboard/onboarding-experimentOutcome
Treatment variant activation rate >= control AND test reaches >= 20 users per variant
Leading Indicators
- Activation rate (control vs treatment)
- Tour completion rate by variant
- Time to activation by variant
- Funnel drop-off delta at tested step
Instructions
-
Run the onboarding-experiment-orchestration drill to analyze your onboarding funnel in PostHog, identify the highest-drop-off surface, form a structured hypothesis, set up a PostHog experiment with feature flag splitting new signups 50/50, and build the treatment variant.
-
Human action required: review the treatment variant, walk through it as a test user, then enable the feature flag.
-
Run the test for 2 weeks. Monitor PostHog Live Events daily (5 min) to confirm both variants receive users and tracking fires correctly. Do not check activation results until the test ends.
-
Run the threshold-engine drill: treatment activation rate >= control, at least 20 users per variant, support ticket rate did not spike > 2x for treatment.
-
If PASS, proceed to Baseline. If marginal (directionally better but < 20 per variant), extend 1 week. If FAIL, investigate session recordings, generate a new hypothesis targeting a different surface, re-run.
Recommendations
Time
6 hours over 2 weeks
Play-specific cost
Free (PostHog free tier: 1M events/mo, 1M feature flag requests/mo; Loops free tier: 1,000 contacts, 4,000 emails/mo)