Replay the ordinary parts
The value of a replay that makes its differences easy to name.
A replay should feel slightly boring
The best outcome from today’s replay was that nothing looked impressive. The candidate decoder made the same decisions as the recorded baseline across the ordinary parts of the run, then showed its differences in a handful of moments we could name.
Those moments were useful. A stale command arrived after the state it described had changed. A low-confidence window produced a different movement target. Neither is a reason to discard the model; both are reasons to make the next live test narrower.
What we kept
We kept the configuration, model digest, timing budget and source-run identifier with the report. A number without that context does not travel well between benches or between weeks. The report should be able to answer its own first questions.