Back to blog
Aug 19
dogfooding
byok
anthropic
governance
engineering-log

Run #6: The Receipts Finally Showed Up, and a Browser We Hadn't Tested In

Run #5 closed with three concrete, unfixed gaps: approval and safety decisions asked for sign-off without showing what they were actually about, run and artifact pages went blank while loading, and the durable audit trail silently dropped most governance decisions even with durable storage attached. All three got fixed before this run — again, not by BinChicken.

What we did manually, before this run

Same discipline as last time: this was direct engineering work, not something BinChicken did to itself.

  • Approval and safety-gate confirmations now show the actual evidence behind the decision — the relevant artifact's content, and the specific reasons a risk classification fired — inside the same confirmation screen, not a separate page you had to already know to go find.
  • Run and artifact pages now show a visible loading state instead of rendering nothing while data is being fetched.
  • The durable audit trail was fixed at its actual source, so a governance decision made through the real, durable path now gets a real, persisted record — not just an in-memory one that vanishes with the browser tab.

Every fix was checked against a real, durable backend (not just an in-memory approximation) before this run started, specifically to make sure the audit-trail fix wasn't just theoretical.

The task

"Improve the BinChicken run-detail page by adding a small 'Last updated' timestamp showing when the run data was most recently refreshed. Use the existing run-detail components and styling. Add appropriate tests. Do not change the workflow engine, AI providers, GitHub integration, governance, authentication, or database schema."

Small, deliberate, and explicitly fenced away from anything sensitive — same shape as Run #5's task.

What happened

One real Anthropic call ran and completed — a few minutes, real BYOK tokens spent — and its interpretation immediately tripped a high-risk safety gate, before the next role even started. The reason given was that the request touched infrastructure or deployment configuration. It didn't; the task explicitly excluded that. This is the third real run in a row where the same kind of false alarm has fired — no longer a one-off worth a footnote, a real pattern worth fixing on its own merits.

The evidence-visibility fix worked. Reviewing the gate, the interpreted specification and the specific reasons behind the classification were both right there in the confirmation screen, exactly as intended — no separate page to hunt for, no blind decision.

Then a real, previously-unseen bug showed up, live, in the middle of reviewing it. In one browser, the confirmation dialog rendered completely unbounded — no size limit, no internal scroll, its own decision buttons pushed off-screen with no way to reach them. The exact same page, at the exact same moment, rendered correctly in a different browser — same content, same code, genuinely different result depending on which browser was open. That comparison is what made the cause findable at all: a rendering difference specific to how one browser handles a particular way of centring the dialog on screen, not a data problem or a logic bug. It was diagnosed and a fix proposed within the same session, but — honestly — it hasn't been confirmed working in the browser that actually broke, only reasoned through and checked in the ones that didn't.

The run was cancelled, deliberately, rather than pushed through. The gate required re-authentication to continue, and between a flag that didn't hold up and a browser that couldn't currently render the decision reliably, cancelling was the right call, not a failure to route around.

The part worth not burying under the bug report

A real governance decision, on a real run, is now durably recorded — for the first time in this series. Every previous run's decisions lived only as long as the browser tab did. This run's safety gate — both the moment it was requested and the moment it was decided — now exists as a real, persisted record, independent of any session. That's the actual claim this product makes about itself, verified against a real decision instead of a test for the first time.

Where this leaves things

  • The false-positive risk classification is now a three-time repeat offender, and the most concrete thing worth fixing next — more so than either bug below.
  • A real, browser-specific rendering failure was found and (probably, not yet confirmed) fixed. Worth naming plainly: none of our previous manual checks caught it, because none of them happened in the specific browser it broke in. That's a real gap in how thoroughly a fix gets checked before it's called done, not just bad luck.
  • Two smaller interface issues came out of the same review: the interpreted specification renders as raw formatting characters instead of readable text, and a classification with an empty reason renders as a bullet point with nothing after it. Neither is fixed yet.
  • Still true from every prior entry: there's no way to stop a run from BinChicken's own interface unless a gate happens to be open offering that choice.

What actually held: real execution, a real (if noisy) safety net, a real audit record surviving past the session that created it, and a human correctly declining to proceed when the tools available to review the decision weren't trustworthy in the moment. Nothing was written to the repository.