Read Experiment Results

Understand Jurni’s current experiment metrics, date-range behavior, attribution differences, and how to compare variants without over-interpreting the data.

Use experiment results to answer the question you wrote before the test started. Start with the primary metric, then use supporting metrics to understand where the customer journey changed.

Do not start by scanning every number for the biggest green percentage.

Read a result in five steps

1. Use the actual test period

   Set the reporting range to the dates you want to evaluate. Exclude known pre-test or post-test periods when they do not belong in the comparison.

2. Start with the primary metric

   Read the metric you chose for the hypothesis—such as CVR, RPS, AOV, or an earlier interaction rate.

3. Compare the observed direction and size

   Which variant is currently ahead on the primary metric, and by how much in observed terms? Treat this as what the data shows so far, not automatic proof of future lift.

4. Use supporting metrics to understand the journey

   Review sessions, clicks, add-to-cart, checkout, orders, revenue, and AOV/RPS where relevant. These can help locate where behavior differed.

5. Check whether anything complicates the result

   Consider traffic/weight changes, routing rules, direct page traffic, product or offer changes, tracking issues, and differences in the compared destinations.

For the strategy layer—what to learn and what to test next—use the Optimization Loop.

Core experiment metrics

Metric	What it tells a marketer	Calculation
Sessions	How many attributed visits reached the variant in the experiment.	Attributed experiment visits assigned to that variant.
Orders	How many attributed purchases came from those visits.	Orders attributed to the variant visits.
Revenue	How much attributed order revenue those visits produced.	Attributed order revenue.
CTR	How often a session included the relevant click-through interaction.	Sessions with the click event ÷ sessions × 100.
CVR	What percentage of sessions produced an order.	Orders ÷ sessions × 100.
Add-to-Cart Rate	What percentage of sessions reached an add-to-cart action.	Sessions with add to cart ÷ sessions × 100.
Checkout Rate	What percentage of sessions initiated checkout.	Sessions with checkout initiated ÷ sessions × 100.
AOV	Average attributed order value among orders.	Revenue ÷ orders.
RPS	How much attributed revenue each session produced on average.	Revenue ÷ sessions.

For the core visit-level interaction rates, repeated actions in the same visit do not make that visit count as multiple converted sessions for the rate.

Which metric should I care about?

Choose based on the hypothesis, not the metric that happens to look best later.

CVR

Use when the main question is:

“Does this experience turn a higher percentage of visitors into buyers?”

Common tests:

• hero message;
• proof placement;
• page story;
• purchase clarity.

RPS

Use when revenue per visitor is the most important overall business outcome.

This can be useful when a change may affect both conversion and the amount purchased.

AOV

Use when the test is specifically about basket size or order value.

Do not use AOV alone to judge a page that changes purchase conversion. A variant can have higher AOV but fewer buyers.

CTR / Add-to-Cart / Checkout Rate

Use these as primary metrics only when the hypothesis is intentionally about that earlier behavior. Otherwise they are often more useful as diagnostics.

Example: CVR rises while AOV falls

Suppose the variant shows:

• higher observed CVR;
• lower observed AOV;
• RPS roughly flat.

A reasonable interpretation is not “the variant won because conversion increased.”

The page may be creating more orders but smaller ones, leaving revenue per session similar. Review the metric that matched your hypothesis and the customer or merchandising change you made.

Example: Add-to-Cart Rate rises but CVR does not

Possible observation:

• more sessions add to cart;
• purchase conversion remains flat or falls.

That tells you the page changed an earlier step in the journey, but it does not tell you why customers failed to finish.

Questions to investigate:

• Was cart/checkout the same across variants?
• Did the offer expectation change?
• Did the variant encourage lower-intent cart additions?
• Was there a product or plan mismatch?

Treat the explanation as a hypothesis until you have more evidence.

Experience analytics and experiment results can differ

An underlying Jurni experience can receive traffic outside the experiment—for example:

• direct page URL;
• another campaign;
• a link that bypasses the experiment Smart Link;
• QA or internal traffic.

That traffic can appear in Experience analytics without being assigned to an experiment variant.

When making an experiment decision, use experiment results rather than assuming the experience’s total session count represents experiment traffic.

Why session counts can differ →

Date ranges matter

Use the same dates when comparing views.

Be especially careful when:

• the experiment started partway through the selected range;
• traffic weights changed;
• routing rules changed;
• the offer or product changed;
• an ad campaign was paused or restarted;
• tracking was fixed during the test.

Historical sessions keep their original attribution. A new weight or routing rule changes future traffic; it does not rewrite the past.

Traffic weights and routing changes

If you alter traffic allocation during the test, record when and why.

A combined full-period result can still be useful, but your team should understand that different parts of the test ran under different allocation conditions.

Routing Rules also mean some traffic can be intentionally directed to one variant rather than randomized through normal weights.

Traffic assignment →

Statistical significance / confidence

A variant being numerically ahead does not automatically mean you have a reliable winner.

Use the confidence/statistical information actually shown in the Jurni experiment workflow available to your account. Do not apply universal rules such as “500 sessions means significant” or “95% is always required” unless the methodology your team is using explicitly defines them.

When a newer confidence/winner-status workflow is available, read the status together with the selected metric, minimum-data conditions, and the actual test context rather than treating the label as permission to stop thinking.

Confidence & winners →

Custom events

Some implementations use mapped downstream events such as quiz steps, email capture, or other custom actions.

These are not standard variant-level metrics throughout every experiment graph. If a custom event is the success condition for your test, agree how it will be measured and read before launching.

If Jurni and another platform disagree

Check:

1. Same date range?
2. Same traffic scope?
3. Is campaign traffic entering through the experiment Smart Link?
4. Does the external platform use the same session/attribution definition?
5. Did consent block one system but not the other?
6. Is the underlying page receiving direct traffic?
7. Are purchases attributed through the same checkout path?

A difference between Jurni, GA4, Meta, Northbeam, and Shopify does not automatically mean one tool is broken.

Understand analytics differences →

A useful AI review prompt

Help me interpret this experiment as a marketer.

Hypothesis: [HYPOTHESIS]
Primary metric: [METRIC]
Test period: [DATES]
Results: [PASTE CURRENT RESULT]
Known traffic / implementation changes: [NOTES]

Separate:
1. What the observed data directly says
2. What it does not prove
3. Plausible interpretations to investigate
4. Confounds / checks
5. Recommended next decision or test

Do not invent statistical confidence, customer intent, or causal certainty.

More examples: Experiment Prompt Library.

Related

• Run your first A/B test
• Optimization Loop
• Metric definitions
• Traffic troubleshooting