A/B Testing Multiple Components

Problem Statement

When a single A/B test spans multiple components, every component must resolve to the same variation for a given visitor. Without consistent Test IDs and Variation IDs, assignments become fragmented, causing misconfigurations, unintended feature interactions, and unreliable results.

Solution

1. Define a single test

• Create one test with a unique Test ID (e.g. TEST-JAN25) to centralize variation management.

• Define all supported variations, each with a distinct Variation ID, e.g.:

  • Variation A: var-a

  • Variation B: var-b

2. Reference the same test in every A/B test instance

When placing multiple A/B test instances on a page or across the site, point each one at the same Test ID and the same Variation IDs:

• A/B Test 1: Test ID TEST-JAN25, variations var-a / var-b

• A/B Test 2: Test ID TEST-JAN25, variations var-a / var-b

A visitor assigned to Variation A in one component then consistently sees Variation A in every other component mapped to that Test ID and Variation ID.

3. Manage the test in Canvas

• Reuse the test: in Canvas, select the same test from Available Tests for each component.

• Set Variant IDs: on the Context tab, assign the specific Variant ID to each variation so the reused test name always resolves to the same Variant ID.

4. Keep configurations identical

All components in the test must share identical variation configurations. Extra or missing variations cause visitors to be reassigned to new variations, breaking consistency. Audit test configurations regularly, especially after configuration changes or feature updates.

Troubleshooting

Verify it works: open the page as a visitor — every component tied to the Test ID shows the same variation; repeat visits keep the same assignment.

Visitors get reassigned to new variations: check for discrepancies in variation lists between components — additional or missing variations trigger reassignment.