Definition
The minimum detectable effect, or MDE, is the effect size at which a specified experiment design achieves its target statistical power under stated assumptions. It depends on sample size, outcome variability or baseline rate, assignment ratio, test choice, and significance level. It describes detection capability rather than the effect a merchant expects to achieve.
Why It Matters
- A low-traffic product-page test may be unable to reliably detect the small improvement needed for its business case.
- A Commerce Intelligence OS should connect test feasibility with the commercial decision before spending weeks collecting insufficient evidence.
How It Works
- Choose a primary outcome and define the smallest improvement that would change the decision. Keep that business threshold separate from statistical sensitivity.
- Estimate eligible independent units and plausible outcome variability from relevant history. Specify the comparison, allocation, significance level, and target power.
- Calculate sensitivity using a method suited to the outcome and design. A two-sample mean calculation is not automatically suitable for conversion proportions, clustered accounts, or sequential monitoring.
- If the MDE exceeds the worthwhile improvement, revise duration, design, or scope before launch. Preserve outcome follow-up and avoid interpreting a nonsignificant result as proof of zero effect.
Ecommerce Example
Context: Illustrative example: a merchant considers a change worth pursuing if conversion rises from 4% to 4.2%.
Recommended move: The business threshold is 0.2 percentage points, equivalent to a 5% relative increase. Calculate whether the available traffic can detect that difference using the chosen design.
Why it matters: If it cannot, the planned test cannot reliably answer the commercial question. The numbers are hypothetical and do not claim a powered sample size.
iKawn Framework
Specify
The iKawn framework states the outcome and worthwhile improvement.
Size
Relate eligible traffic and variability to planned sensitivity.
Design
Match the calculation to assignment and analysis.
Interpret
Report uncertainty against the original decision threshold.
Concise Summary
MDE makes experimental sensitivity explicit. Compare it with the smallest worthwhile improvement and use assumptions appropriate to the actual design.