Provenance Log
Every action that changes the machine's state is recorded with who or what made it, somewhere nobody can quietly alter or withhold.
A test car's log records what it saw, second by second
March 18, 2018 · Tempe, Arizona
Just before 10 p.m., an Uber test SUV driving itself struck and killed a woman pushing a bicycle across N. Mill Avenue. What the machine had done was not left to argument afterwards, because the machine had written it down: its automated driving system recorded when it detected her, how it classified her and what path it predicted each time that changed, the vehicle's own dynamics, and the actions it planned and took. At the NTSB's request, Uber's Advanced Technologies Group played the data back, and investigators built the timeline from the sensor output. First detection came 5.6 seconds before impact; the classification alternated between vehicle, bicycle and other; and 1.2 seconds out, when stopping would have taken more braking than the system was allowed, it began a one-second suppression that held braking off. None of that is testimony. It is a readout.3
Anti-pattern: The Unseen Hand
After the 2019 Key Largo crash the record existed too, and it was the most important evidence in the case: what the car detected, and when. But it sat entirely with the company that built the car, out of reach of the people it concerned. Whatever the reasons, the effect was the same: for years, the people harmed had only their word, and the system's builder had everything else.5
Provenance Log
Therefore: Record every change to the machine's state with who or what made it, keep the record where no single party can alter or withhold it, and make it readable by the people it concerns.
Simplified illustration.
Without
Robot settings
- Max speed
- 8 km/h
- Last changed
- —
Change the setting
Three parties can change this. Use all three.
With the pattern
Robot settings
- Max speed
- 8 km/h
- Changes
- 0 logged
Change log · append-only
- (no entries yet)
Change the setting
Every change writes a line. Try Edit on one.
Every state change carries its author and its time. At Tempe the ADS data held the detection times, the successive classifications and the suppressed brake command, so the sequence could be established rather than argued.
The mechanism
action taken → open it to inspection
- 01Action takenOperator, remote party or machine
- 02Record the actorWho or what, and from where
- 03 · Key momentPut it beyond reachAppend-only, outside the actor's control
- 04Retain itThrough the life of a dispute
- 05Open it to inspectionA record the operator can read
On this page
Design considerations
- Record the actor, not just the change. Who or what made it, from where, and with which software version.1
- Keep the record out of reach of anyone who can change the state. A log that one party to a dispute controls isn't evidence of anything.
- Correct by adding, never by editing. Entries are append-only; a mistake gets a new entry that says so.
- Make it readable by the operator, while they still remember. A log only an investigator reads, years later, protects the institution rather than the operator.
- Keep it as long as a dispute could last. Six months is a floor written for high-risk AI systems, not a guide to how long a claim takes.2
In the field Draft
Observed · Automotive. Uber, Tempe: investigators rebuilt what the car detected and when from its own data — the first detection at 5.6 seconds, the successive reclassifications, and the one-second action suppression that held off braking.3 The same crash is an In the field entry on The Handoff, for the takeover that was never asked for.
Observed · Automotive. Tesla, Key Largo. The collision snapshot was uploaded, acknowledged and marked for deletion in the car within moments, and then sat with the company that built the car for years.5
Observed · Automotive. UN R157 requires automated lane keeping systems to log activation, transition demands, overrides, emergency manoeuvres, collisions, minimum risk manoeuvres and severe failures, each with a timestamp and the software version that produced it.1
Observed · Software. The Post Office Horizon scandal. The Inquiry's first volume puts the number of people prosecuted and convicted on Horizon evidence at approximately 1,000 across the United Kingdom.4
- [Image: Placeholder for a figure from the Key Largo case: the collision snapshot the car uploaded to Tesla's servers, and the acknowledgement sent back to the car.]
Trisha Thadani and Faiz Siddiqui, The Washington Post, 2025 Without it
Observed
Tesla held the car's collision data on its own servers for years while saying it didn't have it.
Related patterns
- Complements · 02The Glass BoxThe Glass Box logs the operator's decision and the reasons for it. This pattern covers every other state change, including the ones no operator made.
- Complements · 04Appropriate FrictionFriction makes a consequential action deliberate. The log is what makes it accountable once it has happened.
- Complements · 09Learning LoopThe log is the record of what the machine did; the loop is what carries what the operator noticed. Neither substitutes for the other, and an investigation usually needs both.
- Complements · 13Operator-to-Operator HandoverA handover is a state change with two actors, and worth recording as one. Who accepted the machine, and in what condition, is exactly the question asked afterwards.
Sources
The data is rarely what's missing. In both the crash and the Horizon case, the records existed. What decides whether a log protects people is who can reach it, and who can change it.
- UN Regulation No. 157 — Automated Lane Keeping Systems (ALKS)Standard
- EU AI Act, Article 19 — Automatically generated logsStandard
- Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, ArizonaIncident
- Post Office Horizon IT Inquiry, Volume 1Incident
- Tesla said it didn't have key data in a fatal crash. Then a hacker found itArticle