Definition
Point-in-time feature integrity means reconstructing model inputs as they could have existed at the decision timestamp. A historical evaluation must not use a later return status, corrected customer profile, or future aggregate as if it were already known. Event time and data availability time can differ, so both may be needed to reproduce the real information boundary.
Why It Matters
- A model may look accurate because its historical inputs contain clues recorded only after the outcome. That performance may disappear in live use.
- A backfilled fact can carry an old event date despite arriving after the decision. Filtering only on event date may therefore be insufficient.
- Predictive commerce needs credible evaluation before a Commerce Intelligence OS uses scores to guide inventory, service, or agent actions.
How It Works
- Assign a decision timestamp to each evaluation row and document when each source fact became usable by the decision system.
- Build historical feature values using only eligible source versions and backward-looking windows. Apply availability cutoffs when late ingestion or later corrections could leak knowledge.
- Separate future outcome labels from input features. Use chronological evaluation splits appropriate to the deployment question and inspect entity overlap.
- Audit sample rows by replaying their source histories. Compare offline and live feature definitions, missing-value handling, and freshness before interpreting model scores.
Ecommerce Example
Context: Illustrative example: a return-risk model is evaluated for the moment an order is placed. The current customer table includes a return recorded two weeks later.
Recommended move: Reconstruct the customer history available at order time and exclude that future return from the inputs. It may belong in a later outcome label under the evaluation definition.
Why it matters: The resulting estimate is less vulnerable to hindsight leakage, but it still requires validation for calibration, data quality, and changing customer behavior.
iKawn Framework
Timestamp
The iKawn framework attaches decision and availability context to commerce facts.
Reconstruct
Build historical inputs from source versions that meet the information cutoff.
Evaluate
Test predictive performance using outcome labels kept separate from the feature pipeline.
Monitor
Compare evaluation assumptions with the information available to live agents.
Concise Summary
Historical model inputs must respect what was knowable then. Event ordering, availability timestamps, and versioned feature logic help prevent hindsight from masquerading as predictive ability.