Definition
A switchback experiment repeatedly assigns an operational unit to treatment or control across time according to a randomized design. It can support evaluation when a shared process makes simultaneous individual assignment impractical. Treatment effects that persist into later blocks can contaminate the comparison.
Why It Matters
- A warehouse dispatch policy changes a shared queue, so assigning separate policies to individual orders may create interference.
- A Commerce Intelligence OS needs an evaluation design that reflects how an operational intervention actually reaches customers.
How It Works
- Define the shared operational unit, eligible time blocks, and decision metric. Check whether the intervention is reversible and whether effects can persist.
- Choose block length and any exclusion or washout rules before the test. Document assumptions about carryover and balance important calendar patterns within the randomized design.
- Log assigned and actual policy states, interruptions, and concurrent changes. Preserve outcomes under the prespecified attribution rule.
- Analyze according to the randomized time-block design, accounting for temporal dependence and carryover assumptions. Do not treat every order as an independently randomized observation.
Ecommerce Example
Context: Illustrative example: a dispatch center compares two queue-priority policies across planned randomized operating blocks.
Recommended move: If one policy leaves a backlog that persists into the next block, revisit the design and carryover assumptions before interpreting the comparison.
Why it matters: The example illustrates evaluation planning; it establishes neither a suitable block duration nor an iKawn performance gain.
iKawn Framework
Map
The iKawn framework identifies where shared resources transmit treatment effects.
Schedule
Record randomized policy assignments across operational time.
Measure
Connect commercial outcomes with the appropriate exposure history.
Interpret
Keep carryover and uncertainty visible in the decision.
Concise Summary
Switchback designs randomize treatment over time for shared operations. Plan for carryover, calendar patterns, and dependence before reading the result.