Making an AI agent trustworthy at volume
I gave Claude the keys to a real codebase and built the governance layer that makes its output safe to ship: typed tools instead of a shell, rules enforced by a guard hook rather than documentation, a definition of done it can't self-certify past, and cost that's measured, not guessed — extracted into a reusable package.
What that bought, measured from its own telemetry: 155 reviewed, merged PRs and 26,693 lines of code in ~110 supervised hours — an estimated 200–355 hours less than building it by hand. The surprise underneath: human review ran a median of 5.8× longer than the model. Plus a real run trace and the two places it broke.