Reliable AI Agents Need a DevOps Feedback Loop

Reliable agent work uses a familiar delivery loop: give an agent bounded work, check the result independently, record what passed, and carry reviewed knowledge forward. That is closer to DevOps than to magic.

I used to describe this as one grand pattern running from cybernetics through Lean, Agile, SRE, Kubernetes, and AI agents. The family resemblance is real. Calling it one proven law was doing too much.

The useful claim is practical: every system that accepts work from independent actors needs a way to constrain attempts, judge results, preserve accepted change, and repair the process after a miss.

Same problem, new workforce

Software teams already know the failure modes.

A change starts with an unclear request. The implementer fills in the gaps. The reviewer sees a different problem. Tests cover the happy path. The release carries an assumption nobody recorded. Six weeks later, another person repeats the same investigation.

AI agents make that loop faster. They do not remove it.

The model can produce a patch in minutes, which means the surrounding system has less time to notice that the request was vague, the evidence was incomplete, or the author graded its own work. Faster output increases the value of a clear boundary and an independent check.

The four moves

I now use plainer names for the coordination pattern.

1. Bound the attempt

Name one behavior, its acceptance criteria, and the files or systems that may change. Put that intent in a durable issue. For my work, that issue is usually a Bead.

The goal is not to predict every implementation step. It is to make drift visible.

2. Check the result

Run deterministic checks, then give a fresh context the same intent, candidate, and evidence. The context that wrote the change should not issue the binding verdict.

Tests answer specific questions. Fresh review asks whether those questions were the right ones and whether the evidence covers the changed work.

3. Record what passed

Git records the exact change. The issue records why the work existed, what acceptance meant, and which evidence supported the result. A durable verdict records what a fresh reviewer actually checked.

This is the point where accepted work becomes inspectable history instead of a claim in chat.

4. Maintain what should be reused

Some results contain a lesson worth carrying forward. Review that lesson, cite the bead or commit that supports it, and add it to an LLM wiki. Link the relevant page from a future bead. If the page proves stale, open work to repair or delete it.

The Knowledge Flywheel is this last move: Beads for the work ledger, an LLM wiki for reviewed knowledge. It does not learn on its own.

What DevOps contributes

DevOps did not make operations disappear. It put development and operations into one delivery system with shared ownership, automation, feedback, and recovery.

The habits transfer cleanly:

SRE adds another useful constraint: reliability work competes with feature work, so teams need an explicit way to decide when failures require repair before more output. The exact metric varies. The discipline does not.

What is different about agents

The workforce changed in ways the delivery system must account for.

Errors can be correlated. Two agents with similar context may produce the same plausible mistake. More reviewers do not create independence if every reviewer sees the same framing and evidence.

Sessions have weak continuity. A new session does not carry the operator's lived memory. The durable parts must live outside the context window in issues, source files, checks, decisions, and reviewed knowledge.

Parallel work is easy to start. Shared files, integration order, and review capacity become the real limits. Spawning another worker does not remove those constraints.

Context can be hostile or stale. Prompt injection, poisoned source material, and outdated instructions turn the input itself into an attack surface. Provenance and least privilege belong in the delivery design.

These differences strengthen the case for a delivery loop. They do not require a new theory of progress.

AgentOps as a public receipt

I built AgentOps around one narrow loop:

intent
  → one bounded implementation
  → fresh validation against the same acceptance
  → durable verdict
  → report and stop

Beads are the preferred intent source. The implementation binds to the original bytes. A fresh context checks the changed scope and evidence. The result is PASS, FAIL, or NOT_PROVEN, with a record another person can inspect.

That core does not own retries, queues, merges, releases, or automatic learning. Those decisions stay with the calling repository. Optional tools can maintain knowledge after the fact, but a wiki page never decides whether the current candidate passed.

The separation is the point. Work state, validation state, and reusable knowledge can support one another without pretending to be the same thing.

The career thread

I came to agent reliability through infrastructure.

Healthcare networks taught me that an elegant change is irrelevant if the operator cannot recover it. Classified and air-gapped systems taught me that missing dependencies surface at the worst possible boundary. Platform engineering taught me to turn a repeated manual path into a paved road with evidence and rollback. SRE made the review and recovery obligations explicit.

Now I apply the same discipline to AI-assisted delivery. The artifact changed. The obligation did not: make the change bounded, observable, reviewable, recoverable, and teachable to the person who owns it next.

The longer arc and receipts are on /about and /work.

A starting point

You do not need a fleet or a control plane to use the pattern.

1. Put the next task in one issue with observable acceptance.
2. Ask one agent to build only that behavior.
3. Run the real checks.
4. Give a fresh context the intent, diff, and evidence.
5. Record the verdict beside the work.
6. Add a wiki page only if the lesson will change another decision.

That loop will expose the next bottleneck. It may be planning, tests, review capacity, integration, or missing context. Fix the bottleneck you can see before adding more agents.

Same problem, new workforce. The output got cheaper. Judgment and memory still have to be engineered.

Related

• CDLC: the seven-phase lifecycle for maintained agent context
• The Knowledge Flywheel: Beads plus an LLM wiki
• The Validation Bottleneck: why judgment limits the loop
• 12-Factor AgentOps: the vendor-neutral operating principles