The most useful thing a coding agent can remember is often not a fact. It is a trail. Which files it read. Which tests it ran. Which patch it tried. Which assumption broke. Which command finally proved the fix.

That isn't a diary. It's closer to a black-box recorder — the thing you go pull after the run to find out what the agent did, command by command, separately from what it later claimed in the summary.

Earlier posts in this series were about claims: canon, retrieval candidates, relationship state. This one is about evidence — the structured record of what happened, before any of it gets summarized into a sentence.

A coding-agent black box recording commands, file changes, test results, errors, and abandoned plans for later query.

The transcript is evidence

The ordinary pain that motivates this layer: an agent gets handed a task, fumbles it, fixes something tangential, gets handed the same kind of task next week, and proceeds to repeat the exact failed approach because the previous run's trace either vanished or got summarized into mush at compaction time.

This is the failure mode AgentMemory-style capture is trying to solve. Hermes does something similar with session traces. Pi keeps sessions under ~/.pi/agent/sessions/. Belayer's whole orchestration model assumes a downstream agent can pull up a prior agent's tool calls and read them. The shape varies, but the move is the same: capture tool events, decisions, failures, diffs, tests, and provenance as structured records so the next run can read them.

Critical distinction: capture is not retrieval. Stuffing every prior tool call into the next prompt is the laziest version of this. The point of the black box isn't to replay everything — it's to have the recording when the next agent needs to ask a specific question, like "did we already try the regex fix and why did it fail?" That kind of query doesn't need the whole transcript. It needs a queryable record.

The analogy I keep using is the flight recorder versus the flight manual. The black box tells you what happened during the flight: altitude, alarms, pilot inputs, impact. It does not tell future pilots the approved procedure by itself. The manual is canon. The recorder is evidence that may justify changing the manual after review.

What belongs in the black box

The fields I keep finding useful, in roughly this order of frequency:

  • Command executed (full argv, working directory, exit code)
  • Files read and changed (with diffs for the changed ones)
  • Tests run and results (including which ones were skipped and why)
  • Errors encountered (stack trace, not just the summary line)
  • Assumptions invalidated (the thing the agent thought was true, the moment it found out it wasn't)
  • Plans abandoned (with the reason — this is the highest-signal field)
  • Diff/PR produced (and link to it)
  • Model and harness used (Pi 0.75.3 vs Claude Code, Sonnet 4.6 vs Opus 4.7 — drift matters)
  • Issue/spec/task that triggered the work
  • Reviewer feedback and whether it was addressed

This is operational memory, not canon. None of these are claims about the world. They're claims about what happened during this run. They feed canon — eventually — but they're not canon themselves.

Failures should be remembered differently

High-signal records: failed attempts and abandoned plans. Low-signal records: every successful tool call.

The reason is asymmetric: the successful call resolved into the diff, which is the durable artifact. The failure resolved into a dead end that disappears unless you write it down. The next agent that tries the same approach will re-discover the dead end the same way unless the black box surfaces it.

The concrete scenario I've watched several times:

  • Agent A runs the Playwright test, port conflict with a leftover dev server, test fails for the wrong reason. Agent A debugs the port conflict, kills the process, test passes, ships.
  • Agent B picks up a similar task next week. Same port conflict. Same failed test. No record of Agent A's fix. Agent B spends an hour re-deriving the port problem from scratch.

The black box should mean Agent B's first tool call is "search prior runs for this test failure" and the first hit is Agent A's recovery. Not because the memory is clever — because the recording exists.

These failures should still expire. A port conflict from six months ago might not be true anymore (maybe the test harness changed). The black box layer needs the same lifecycle primitives the semantic recall layer needs: decay on stale records, reinforcement when a pattern keeps recurring, promotion to canon when a fix becomes durable enough that it should live in AGENTS.md instead of in the trace store.

A good black-box hit should look boringly specific:

query: "did we already try fixing the receipt parser by changing the discount regex?"

hit:
- task: receipt parser regression, 2026-05-28
- attempted patch: widened `/discount|promo/i` to include `rebate`
- result: unit tests passed, fixture test failed on tax line parsing
- rejected hypothesis: discount label was not the real bug
- final recovery: parser needed line-type precedence before amount extraction
- evidence: diff, failing command, final passing command

That is the useful version of memory: not "I vaguely remember regex was involved," but a little crash report with enough structure for the next agent to skip the dead end and verify the current repo still behaves the same way.

The write-back path matters

The tempting failure mode: AgentMemory captures everything, the agent treats its own captured traces as authoritative, and you've quietly built a second source of truth about the codebase that nobody reviewed. The trace store starts claiming "this repo uses pnpm" because the last fifty commands all ran pnpm install, and now you have two things calling themselves canon — the wiki and the trace store — and they're allowed to disagree.

This is the "two canonical stores = no canonical store" rule from the Agent memory isn't one thing hub. The trace store is not canon. It's evidence. The path forward is:

  1. Black box captures evidence (passively, as a side effect of the run).
  2. The reflective layer (Hindsight-style, or just a periodic agent task) proposes a durable lesson.
  3. Review gate decides whether it goes into canon.
  4. Canon doc updates.
  5. Future agents read the reviewed rule, not the raw trace.

This is the same governance shape as Who gets to write truth? — the trace store is just one of the layers feeding into the proposal step.

What the black box isn't

Worth being explicit because the framing gets misread fast:

  • Not "raw transcripts > summaries." Raw traces are evidence; summaries are working views. Both belong. The mistake is letting summaries become evidence.
  • Not "stuff every tool call into future context." The black box is queryable storage. It is not prompt-stuffing material.
  • Not a replacement for docs. It feeds docs. If a pattern recurs often enough to belong in AGENTS.md, it should get promoted there; the trace store stops being the source of truth for that pattern the moment the promotion happens.
  • Not a passive blob. The useful version is queryable and structured — JSONL events, indexed by task and file path, not a 200MB transcript dump.

Outcomes

  • Tool traces are memory too. The fact that they're not claims about the world doesn't make them less load-bearing — they're how you avoid repeating last week's mistakes.
  • Capture is not retrieval. Record everything; load only what the next query asks for.
  • The black box preserves evidence, prevents repeated failures, and feeds reviewed canon when a lesson becomes durable. It does not become canon on its own.
  • The next post is the routing argument — why the explore, climb, and summit phases of a coding task need different projections of all of this.