Definition
For a binary return-risk flag, precision is the share of flagged orders that later meet the defined return outcome. Recall is the share of all orders meeting that outcome that were flagged. These measures evaluate a thresholded prediction; they do not establish why a customer returned an item or whether an intervention would prevent it.
Why It Matters
- Overall accuracy can conceal a model that rarely identifies the outcome a team needs to address.
- Too many low-value alerts can consume service capacity, while a narrow alert policy can miss relevant cases.
- A Commerce Intelligence OS needs evaluation tied to the intended support action rather than an isolated model score.
How It Works
- Define the prediction moment, eligible order population, return outcome, and follow-up horizon. Use historical inputs available at prediction time and sufficiently mature labels.
- At each candidate threshold, count true positives, false positives, and false negatives. Calculate precision and recall with explicit handling for empty denominators.
- Compare thresholds on held-out data and report alert volume alongside category-level results. Keep the evaluation return prevalence visible when comparing populations.
- Choose a support workflow using capacity, customer impact, and intervention evidence. Evaluate any preventive action separately because accurate risk prediction does not prove that acting on it creates benefit.
Ecommerce Example
Context: Illustrative example: among 1,000 mature orders, 100 are returned. A model flags 80 orders, of which 40 are returned.
Recommended move: Precision is 40 divided by 80, or 50%. Recall is 40 divided by 100, or 40%; the model misses 60 returned orders.
Why it matters: The team can assess whether reviewing 80 orders is feasible, then test useful assistance separately. These hypothetical figures do not justify denying returns or claim an iKawn model result.
iKawn Framework
Define
The iKawn framework connects a risk label with its event and maturity horizon.
Evaluate
Show correct alerts, incorrect alerts, and missed outcomes.
Select
Match a candidate threshold to a specific assistance workflow.
Validate
Measure the actual effect of assistance beyond prediction accuracy.
Concise Summary
Precision describes alert quality; recall describes captured outcomes. Evaluate both with alert volume and mature labels, then test whether the resulting action helps.