Back to blog
Aug 19
dogfooding
byok
anthropic
governance
engineering-log

Run #5: A Real Spec, Real Parallel Steps, and a Rejection We Couldn't See Enough to Avoid

Run #4 closed with two concrete, unfixed defects: a backend bug that could make a run fail to start for no reason a user could diagnose, and a confirmation dialog that could render off-screen with no way to scroll it into view. Both got fixed by hand before this run — not by BinChicken. This entry covers that work, then a run that went further and cleaner than any before it, and one more real interface gap it took a real human decision to surface.

What we did manually, before this run

Again: none of this was done by BinChicken.

  • Replaced the identifier scheme behind runs, steps, and related records with one that's actually safe across separate server processes, not just within one. The old scheme could — and once did — produce the same id twice across two different process starts, which is what failed the very first attempt at Run #4. Added automated tests that specifically simulate two independent fresh processes and confirm they can never collide; verified those tests actually catch the old behaviour before trusting them.
  • Fixed the confirmation dialog so it stays within the screen and scrolls internally when its content is long, keeping its action buttons reachable no matter how much text is above them.

Both were verified with the project's full automated check suite before anything else happened.

The task

This time, a real one:

"Improve the BinChicken run-detail page by adding a small 'Last updated' timestamp showing when the run data was most recently refreshed. Use the existing run-detail components and styling. Add appropriate tests. Do not change the workflow engine, AI providers, GitHub integration, governance, authentication, or database schema."

Small, concrete, and explicitly fenced away from anything sensitive — a good test of whether a legitimate, narrow request behaves differently from Run #4's accidental garbage input.

What happened

Four roles made real calls to Anthropic, on the account's own key, and all four completed cleanly. Scope and architecture ran first, each taking a few minutes. Once architecture finished, the two roles that only depend on it — UX review and security review — started at the same moment and ran genuinely in parallel, each taking a few minutes on its own. Tens of thousands of real input tokens went into these four calls combined, on a real task this time, not an error message.

No false alarm from the safety gate. Run #4's confirmation dialog fired on a false positive — descriptive text about existing security-adjacent code, misread as a proposal to change it. This run's architecture and security steps stayed on-topic for a small UI change, and nothing tripped the gate. One clean pass isn't proof the underlying issue is gone, but it's the first real evidence a narrow, on-topic specification doesn't automatically hit it.

The run correctly stopped at a human approval gate — and the decision that gate got was itself correct, for a reason worth sitting with. Once the security review finished, BinChicken paused and asked a human to sign off on it before continuing, exactly as it's supposed to. The confirmation dialog showed the gate's name and a one-line description. It did not show the security review itself, or a way to get to it from inside the dialog. The reviewer rejected it, with the reason recorded verbatim in the audit trail: the evidence needed to make an informed decision wasn't visible.

That rejection is a governance success, not a failure, and it's worth being precise about why. BinChicken didn't quietly wave the step through because a human clicked something. It durably recorded a real rejection, permanently blocked every step downstream of it, and stopped — correctly refusing to proceed without a decision it could stand behind. The infrastructure did exactly its job. The gap is that the interface asked for a decision without giving the reviewer what they needed to make one with confidence, and a second, related gap made checking the evidence directly harder than it should have been: the page that shows a produced artifact has no loading indicator, so following the link to go read it, mid-review, produced a blank screen for long enough to look broken before the content finally appeared.

Where this leaves things

Two real interface gaps came out of this run, both about the same underlying problem — asking for a decision without showing the thing the decision is about:

  • Approval and safety-gate confirmations don't surface what they're actually asking you to approve. The record connecting a gate to the artifact it's gating already exists; the confirmation screen just doesn't use it. A reviewer currently has to already know to go find the artifact separately, in a different part of the interface, before coming back to decide.
  • The page that shows a produced artifact has no loading state. A slow load — and this one is slower than most, since showing one artifact means pulling in a fair amount of its run's surrounding context first — reads as broken rather than in progress, at exactly the moment a reviewer is trying to check something before making a real decision.

Neither is fixed yet. Both are directly responsible for the shape this run's ending took: a correct rejection that a better interface would have made unnecessary. Also still true, and not new to this run: there's still no way to cancel a run from BinChicken's own interface if one is ever genuinely unwanted rather than correctly stopped — every terminated run in this series so far, including the correct ones, has ended through a human decision the product recorded faithfully, never through a control the product itself offers.

What actually held: a real specification ran end-to-end through real parallel execution without incident, no false alarm interrupted legitimate work, and when a human couldn't confidently approve something, the system believed them completely — no partial progress, no silent continuation, nothing written to the repository. The next run needs an approval screen that shows its own evidence.