After Code Stopped Being Scarce (Part II): From AI Writing Code to AI Taking Part in Delivery

Before I knew it, a year had passed since I left my previous employer. I have learned a lot this year, and I have been through many things I could not have imagined before.

Many people find my choice hard to understand, and I find it hard to explain precisely myself. In the end, I just wanted to do something interesting.

This article is about one of those things: a new delivery workflow we have been trying to build for the AI era. I wrote the first version in August. Recently quite a few people have come to us asking about this approach, so I have updated it here with what we have put into practice over the past few months.

Recently, the well-known open-source developer yetone also published his workflow. When we compared the two, we found that the core ideas largely converge: different paths arriving at the same place. Our financial business additionally requires certain human reviews and tests that simply cannot be left out, and this also determines how far we can push automation. We hand the execution that suits automation to Agents, and leave key business judgments and release authorization to people.

The core of this workflow today is Claude Code and Claude Opus 5.5. Codex takes part in troubleshooting, development and acceptance, and Grok Bot is responsible for independent CR and verification. Rules, steps and evidence all stay in the engineering system, so other executors can pick them up and keep going. But being able to take over is not the same as achieving the same results. Based on our experience so far, switching to other models and Agents may not achieve the same results.

What we maintain is a financial business system spanning multiple repositories, and a single requirement may pass through the frontend, the middle platform, domain services and external channels. Here, getting a model to write a piece of code is not hard. What is hard is finding the right place to change, holding to the business constraints, and then proving that the result is still correct after the change.

Based on rough counts from the records of our three main repositories from August to October, approximately 1,900 change PRs went into a release, of which approximately 1,800 can be traced to successful production deployments. Over the same period, snapshots of the business and test source code in delivered versions show approximately 630,000 lines added and approximately 60,000 lines removed, including approximately 290,000 lines of added test source code. Code volume is compared between the start-of-period and end-of-period snapshots, not summed across the changes in each PR. These numbers record the actual scale of delivery. Nearly half of the added source code is tests, which reflects our investment in verification.

The diagram below shows the full picture of our delivery today.

What the diagram shows is the division of labor and the feedback relationships we have built around delivery. At the starting point, people define the goals and acceptance criteria. Development, review, fixes and re-review are pushed forward continuously by Agents. Changes are first verified in the test environment, and then people authorize the merge and release. All stages share the same rules, tools and records.

Keeping people at these points is a deliberate trade-off on our part. If a judgment about fund semantics, permission boundaries or release timing goes wrong, it may affect users’ funds, and rolling back the code may not undo transactions that have already happened. The basis for these judgments is often not in the repository either. Agents can lay out the evidence and flag the risks. Whether the acceptance criteria hold, whether a business trade-off is acceptable, and when to go into production are decided by people, and people are accountable for them.

Don’t switch platforms; organize around the existing engineering ground

When moving toward Agents, a natural impulse is to switch platforms: move requirements onto a nicer-looking board, then hook up a console that can dispatch tasks. We looked at several solutions that tie together Issues, local Agents and execution status, and in the end we kept GitHub Issues and continued to use Claude Code and Codex locally.

These executors work in the same place as engineers, using the same repositories, the same CLIs, and the same tests and deployment scripts. Requirements, decisions, reviews and CI results are already in GitHub. Bringing in another platform to replicate these facts means dealing with syncing and conflicts.

Their strength lies in connecting reading the scene with taking action. Once, a UAT release stalled at Progress Deadline Exceeded. Under the old division of labor, it would have been handed off from the on-call colleague to the platform colleague and then on to the backend. That time, we handed the repository, the target environment and the failure scene directly to Codex. It dug down through the Deployment, the Pods and the startup logs, confirmed that the cluster was fine, and found that the real cause was a callback handler that Spring had not loaded. It completed the fix and verification and opened a PR; another model reviewed it independently, and finally an engineer did the CR.

So when we add a piece of automation, we usually start by providing a reliable execution entry point and clear rules, rather than first designing a dedicated flow for every exception branch. The prerequisite is that the Agent’s judgment is subject to verification, and that tool permissions and task scope are clear.

The division of labor can also be adjusted per task. In October, for one cross-repository review, fixes were handed to Claude and acceptance to Codex, with both using the same set of rules and the same batch of PRs.

Rules, steps and state all stay in the engineering system

When people collaborate with each other, a lot of background can be filled in with a phrase like “that issue from last time.” Agents don’t have that kind of tacit understanding; open a new session, and the earlier background may not come along. Without solving this, engineers turn into couriers of context.

Our approach is called X as Code (XaC): rules can be versioned, steps can be executed, and facts and state stay in the engineering system. Issues and PRs record requirements and decisions, AGENTS.md and topic-specific specs record constraints, Skills and scripts carry repeated steps, CI performs deterministic checks, and infrastructure is described in Terraform. Business trade-offs are still prose, such as how to handle a case where the channel has succeeded but the local side has failed. This kind of prose is written in Issues or code comments, where it can be found and cited, and where it can be noticed when it goes stale.

The CLAUDE.md in our main repositories links directly to the AGENTS.md in the same directory. If you maintain a separate copy for each executor, the rules will diverge sooner or later.

Where the rules live also determines how much has to be read. Early on, every time something went wrong, we added an entry to AGENTS.md. After a few rounds, even changing a small frontend component meant first reading a pile of background on funds and transactions, and the key rules ended up buried instead. Later we switched to reading layer by layer according to the task.

The Agent first reads the repository-wide constraints and the work entry point; whichever directory the change goes into, it reads that directory’s boundaries. Then, through a topic index, it goes only into the amount, transaction, tenant or contract topics involved in this change. The bottom layer is code, tests and runtime evidence, used to check the understanding built in the layers above. When verifying behavior that is already implemented, code takes precedence over comments, and comments take precedence over documentation.

So documentation only records what cannot be read from the code, including roles, boundaries, fund semantics, design trade-offs and the pitfalls we have run into, and does not copy class lists or field tables. Documentation that copies code structure will go stale sooner or later, and Agents and new colleagues alike will follow it to wrong conclusions. We have cleaned up documents whose descriptions did not match the actual implementation, and we have also moved unused Skills out of the loading directory. Context does not need to be full; it just needs to be accurate enough.

The rules in our main repositories put “simplicity first” as the first item: if something can be reused, don’t add a new abstraction, and before adding new state, prove that it represents an independent business meaning. In review we often ask: if we delete this piece of design, can the requirement still be met? If so, delete it. AI easily turns a small requirement into a refactor done along the way. This rule keeps the size of deliveries in check, and also keeps in check what the next person taking over has to understand.

The rules are updated as practice evolves, and reviewers also have to state which paths they actually checked.

Independent review, and follow-up that doesn’t need people to chase it

The biggest change since August is that CR and independent verification have been handed to Grok Bot. Implementers still test their own work, CI still performs deterministic checks, and people are responsible for key business judgments.

Independent review is needed because an implementer’s self-testing tends to carry over the assumptions made during implementation. Grok Bot starts from the requirements and the repository rules and checks the parts the implementer has not proven, which often lie outside the diff. For example, for a withdrawal action, you can’t just check whether the submission endpoint is idempotent; you also have to follow the call chain to compensation jobs, manual retries and Webhooks: if the channel has already succeeded and the local write has failed, what happens if it runs again?

The verification method is decided per task. Sometimes it checks code, tests and CI evidence; sometimes it actually runs endpoints and pages in the test environment and checks the deployment results. The report has to distinguish what was actually executed, what was only checked against existing evidence, and what has not been verified yet. Independent review, CI and human judgment each carry part of the responsibility for verification. After receiving feedback, the implementer has to go back and check it against the code: fix what holds up, explain the reasons for what is not adopted, and hand decisions involving funds, permissions or business semantics to people.

After bringing in Grok Bot, one kind of work was still left with people: waiting for review, reading the feedback, fixing, waiting for CI, and then checking again for new issues. If every step needs a person to come back and say “continue,” the cost of waiting and context switching is still there. PR watch handles continuous follow-up, and Autopilot pushes the PR to a mergeable state within the current session. Each round reads the latest state from GitHub, handles in-scope conflicts, review comments and CI failures, and once the fixes are done, hands it back to review for re-review. Once the PR reaches a mergeable state, merging and release follow human authorization.

Releases are matched to runtime results; problems flow back into engineering assets

To go from Issue to production, a change must first be verified in the test environment, and then a person authorizes the release. Grok Bot performs test environment verification according to the task, and its conclusions have to match the version actually deployed and the results actually observed at runtime. Runtime evidence from before and after release is checked by the corresponding Skills. When the runtime environment or the database cannot be read, the conclusion is “missing evidence”; you cannot declare something releasable just because the code matches.

What “green” proves also has to be made clear. CI passing only means the preset checks have passed; it does not show that the external channels, page interactions and the production path are all fine, and it is even less the same thing as business acceptance. We forbid making tests green by deleting assertions, loosening conditions or skipping failures. To fix a bug, first write a test case that fails under the old implementation, proving that the test really can catch the original error.

Early on, every incident added another paragraph to the rules; the file read more and more like a written self-criticism, yet it rarely stopped similar problems. Now, when we find a problem, we first decide what form it should be kept in: what can be reliably reproduced is kept as a regression test, what can be mechanically identified is turned into a check, what can only be explained with business context is written into code comments, cross-module constraints go into topic-specific specs, and flows that are executed repeatedly become Skills.

Development verification uses isolated environments, and parallel tasks are scheduled according to their dependencies on the shared database and the external channel sandboxes.

Where we stand now

Engineering assets need to be portable, and there is also a threshold for executor capability. A single source of rules, callable Skills and evidence that matches versions let Claude Code and Codex divide work on the same batch of PRs, and also let new executors plug in later. These assets determine the starting point for taking over; how deep one can go still depends on the executor itself. We currently keep our main line on Claude Code and Opus 5.5, based on our actual usage so far.

People’s attention goes to three things: whether the goals and acceptance criteria are right, whether the key business trade-offs are right, and when changes are allowed into production. Everything else is changing every month. We will keep handing to Agents the steps that need people to relay, chase or repeatedly check, and see how much further we can go. This is also what makes this whole thing interesting to me.