Metrics & Results
What NavonaAI tracks during experiments and how winners are determined
Experiment Metrics & Results
When you run an A/B test, NavonaAI tracks detailed metrics for each variant so you can compare their performance side by side.
Summary Cards
The top of the results page shows three headline numbers:
| Metric | Formula |
|---|---|
| Total Sample Size (Carts) | Group A carts + Group B carts |
| Orders Difference | Group B orders - Group A orders |
| Revenue Difference | Group B revenue - Group A revenue |
These give you an immediate sense of which variant is winning and by how much.
Metrics Tracked Per Variant
The detailed comparison table shows these metrics for each variant:
| Metric | Description | Formula |
|---|---|---|
| Sample Size (Carts) | Number of carts assigned to this variant | Direct count |
| Orders | Carts that resulted in a completed purchase | Direct count |
| Average Order Value | Average revenue per order | Revenue / Orders |
| Revenue | Total order value attributed to this variant | Sum of order totals |
| Cart to Order Rate | Percentage of carts that became orders | (Orders / Sample Size) x 100 |
| Cart Abandonment Rate | Percentage of carts that never became orders (lower is better) | ((Sample Size - Orders) / Sample Size) x 100 |
Each metric also shows a Difference column — the percentage change from Group A to Group B. For Cart Abandonment Rate, the sign is inverted because lower is better (a decrease shows as positive/green).
Daily Performance
The daily performance table tracks the selected goal KPI for each variant on each day of the experiment. This helps you spot:
- Trends — Is one variant consistently winning, or did it spike on one day?
- Convergence — Are the variants getting closer or further apart over time?
- Anomalies — Did an external event (sale, ad campaign) affect one day disproportionately?
Click any row to see the full daily breakdown for that date.
Choosing a Goal KPI
When creating an experiment, you select the primary KPI used to determine the winner:
| KPI | Best For |
|---|---|
| Conversion rate (Cart to Order Rate) | Maximizing the share of carts that become orders |
| Abandonment rate (Cart Abandonment Rate) | Minimizing lost carts (lower is better) |
Which KPI should I choose? For most stores, conversion rate is the best starting point — it directly measures how effectively each variant turns browsers into buyers.
How Winners Are Determined
When an experiment completes:
- NavonaAI aggregates all events from carts assigned to each variant during the experiment window
- The primary goal KPI is compared between the two variants
- The variant that performed better on the chosen KPI is identified as the winner
The winner can then be promoted to apply its configuration as your store's default settings.
If both variants show identical performance on the primary KPI, the experiment is marked as Inconclusive. You can review the detailed metrics to decide whether to re-run with a larger sample or pick a variant manually.