Record unexpected events during a live experiment

Unexpected events belong in the experiment record because they may explain an apparent improvement or decline.

A practical procedure

1. Keep a timestamped note for outages, surprise promotions, audience changes and product substitutions.

2. Mark which observation windows were affected and which remained comparable.

3. Report the result both with its context and with any excluded windows clearly identified.

Worked example: a fictional session

A fictional opening trial receives an unplanned referral from another account; the report records that exposure rather than crediting the script alone.

Check the limits

Excluding inconvenient events after seeing the result can distort the comparison.