Definition
Rolling-origin evaluation repeatedly trains a forecasting method using information preceding a cutoff, predicts subsequent periods, and advances the cutoff. It simulates a series of historical forecast decisions. Teams can use expanding or fixed-length training windows and evaluate the horizons relevant to the planning task.
Why It Matters
- One quiet test month may not represent the conditions faced by a replenishment team during promotions or seasonal changes.
- A buyer placing a four-week order needs evidence about that horizon, not only tomorrow's forecast.
- A Commerce Intelligence OS should preserve the planning date behind each score so a reviewer can reproduce the decision context.
How It Works
- Choose the initial training window, cutoff schedule, refit frequency, and forecast horizons before comparing methods.
- At each origin, fit preprocessing and the model using eligible historical data. Reconstruct the information available then, including the version of promotion plans.
- Store forecasts by SKU, location, origin, and horizon. Score competing methods on identical eligible observations and report results separately by lead time.
- Inspect difficult periods and retain a final untouched period when repeated tuning uses the rolling results. Overlapping test windows create dependent errors, so avoid treating every score as an independent observation.
Ecommerce Example
Context: Illustrative example: a retailer forecasts the next four weeks at every month-end using all history available at that date.
Recommended move: The reviewer compares one-week and four-week errors separately, then inspects whether a model that looks strong overall fails around category launches.
Why it matters: The resulting evidence supports a planning review; it does not guarantee future accuracy. This is an evaluation design example rather than a deployed iKawn result.
iKawn Framework
Schedule
The iKawn framework records when a forecast informs a buying decision.
Replay
Recreate successive historical planning conditions.
Compare
Align methods by assortment and operational horizon.
Review
Connect weak evaluation periods with specific planning exceptions.
Concise Summary
Rolling origins test repeated historical forecast decisions. Match the evaluation horizon and refit schedule to the actual planning workflow.