Question 1

How do I read the results table?

Accepted Answer

Each row is one metric. You see A and B, their ranges (confidence intervals), the difference, the p-value, and whether it is significant. Highlighted rows mean A and B differ.

Question 2

What do the confidence intervals mean?

Accepted Answer

They are the 'likely range' around a number. At 95%, the true value should fall in that range most of the time. If the difference range does not cross 0, we mark it significant.

Question 3

What does the p-value represent?

Accepted Answer

It tells how surprising the result would be if there were no real difference. Smaller than 0.05 (for a 95% level) means it is statistically significant.

Question 4

What is Rel. Change and the mini error bar?

Accepted Answer

Rel. Change is the percent change of B vs A. The small bar shows its range; colored = significant, gray = not.

Question 5

Can I use unequal sample sizes for A and B?

Accepted Answer

Yes. Different sample sizes are fine. Equal sizes usually give more power.

Question 6

What inputs do I need for proportion vs. continuous metrics?

Accepted Answer

For proportions (yes/no outcomes like conversion), enter sample size and conversion count or rate. For continuous metrics (like revenue or time), enter sample size, mean, and standard deviation for each group.

A/B Test Results Visualizer

Metrics

Results

Conversion Rate

Common questions