Getting Started with Vibe Coding: An AgentOps Quickstart

Vibe coding is the entry. AgentOps is how it stays reliable across more than one session. This article is the principle-level quickstart; the full operating loop is at /method.

Andrej Karpathy named it: let the AI write code while you direct. That works for a one-off. It falls apart by session three.

The failures are operational. Agents forget what they tried. Plans pass review and ship bugs anyway; 18% of my tasks needed a second pass the week I first measured it. Lessons evaporate between sessions. Longer prompts make this worse past about 40% context utilization. The gap between generating output and trusting it is what these practices close. What helps: fewer tokens, each paid for by intent, evidence, or a constraint that changes the next run.

AgentOps applies practices software teams already trust (BDD, DDD, TDD, and independent review) to the workflow coding agents run. It keeps a file-backed record of intent, candidate scope, evidence, and verdict so another context can inspect what happened.

The Four Layers

Each layer solves a different failure mode.

AgentOps run artifacts live under .agents/ as inspectable files. Work state lives in Beads, and reusable knowledge lives wherever the repository keeps its reviewed LLM wiki. No hosted control plane is required. The skills are runtime-neutral across Claude Code, Codex CLI, Cursor, and OpenCode.

The Operating Loop

One invocation handles one bounded experiment:

fixed intent
  → Plan
  → one bounded Implement pass
  → fresh Validate against the same acceptance and evidence
  → PASS, FAIL, or NOT_PROVEN
  → report and stop

A few rules carry the rest:

• Behavior is the unit of work, not a layer. A slice cuts vertically through whatever layers it needs to demonstrate one Given/When/Then.
• The first failing test is the contract. Code without a failing test has no acceptance surface; the agent can't know when it's done.
• Parallelism is optional and explicitly owned. RPI does not spawn a wave. If a caller uses /swarm, default to sequential and parallelize only disjoint write scopes.
• Context crosses boundaries as artifacts, not as accumulated chat.

Your First Cycle

Three commands. About fifteen minutes.

1. Install

# Claude Code
claude plugin marketplace add boshu2/agentops
claude plugin install agentops@agentops-marketplace

# Codex
codex plugin marketplace add boshu2/agentops
codex plugin add agentops@agentops-marketplace

# Beads Rust work tracker
curl -fsSL "https://raw.githubusercontent.com/Dicklesworthstone/beads_rust/main/install.sh" | bash

Then install the ao CLI for repo seeding and health checks:

brew tap boshu2/agentops https://github.com/boshu2/homebrew-agentops
brew install agentops
ao doctor

ao doctor is the canonical health check. Non-zero exit means a real problem.

2. Seed the repo

cd <your-repo>
ao quickstart

This creates .agents/ and prints the single next action for your state. Re-runnable. Idempotent.

3. Run one full loop

/rpi "a small goal"

Pick something you'd normally finish in 30 to 60 minutes: one endpoint, one component, one bug. /rpi runs the current core once:

Plan → Implement → fresh Validate → report

The expected receipt is the resolved intent, the exact candidate and changed paths, check evidence, a fresh verdict, and the final report. Depending on the runtime, those references may be returned directly or stored under .agents/; do not assume one legacy run directory.

You can invoke Plan, Implement, and Validate separately. Preserve the exact intent across phases, and run Validate in a context distinct from the author session.

Behavioral Discipline

The skills enforce four habits.

A concrete example. User says: "Make search faster." The default agent picks a meaning (latency? throughput? perceived?), adds caching, and ships a larger patch than the question justified. The disciplined agent asks which metric, picks the smallest change that moves it, and verifies against the metric that actually mattered.

The Knowledge Trail

Most observations are noise. Keep work bookkeeping separate from reusable guidance.

No session promotes itself automatically. A later Learn pass can propose maintenance, but a person or repository gate decides whether the wiki or a rule changes.

Validate Before You Ship

The core rule is simple: the author context does not issue the binding verdict.

/pre-mortem: simulate failures before implementing. Runs before code exists. Catches things like "this CRD breaks backward compatibility" or "the reconciler infinite-loops if the finalizer isn't idempotent."

/validate: judge the fixed intent, exact candidate, changed paths, and check evidence from a fresh context. It returns PASS, FAIL, or NOT_PROVEN.

/council: optional extra perspectives for a contested judgment. Several judges can share the same blind spot, so council advice does not replace the fresh Validate verdict or deterministic checks.

> /council --mixed validate this PR

[council] sealed evidence packet → 6 judges across Claude Code and Codex CLI
[claude/judge-1] WARN, rate limiting missing on /login
[codex/judge-1]  WARN, token bucket refill lacks jitter under burst
Consensus: WARN, fix /login rate limit and add refill jitter before shipping

Common Mistakes

I made all of these.

1. Treating every .agents/ artifact as reusable knowledge. Run files explain one experiment. Put work in Beads and promote only reviewed, source-linked lessons into the wiki.

2. Parallelizing shared files. Optional orchestration amplifies a bad ownership plan. Use one writer by default; reserve or sequence shared paths before starting a wave.

3. Letting observations promote themselves. A note becomes durable guidance only after review shows that it generalizes and its source remains valid.

4. Skipping the first failing test. Without it, the slice has no acceptance surface. The agent will declare victory on code that doesn't compile. The overnight run in devlog 5 closed 4 tasks that way; 2 were actually done.

5. One giant prompt. As noted up top, context past the threshold degrades reliability. The 40% Rule in devlog 2 came out of this mistake. Break work into slices with clean boundaries; let the compiler hand each slice the bounded packet it needs.

What's Next

The full method is at /method. The doctrine, including why this works the way it does, is at 12-Factor AgentOps.

Try It

# Install
claude plugin marketplace add boshu2/agentops
claude plugin install agentops@agentops-marketplace
brew tap boshu2/agentops https://github.com/boshu2/homebrew-agentops
brew install agentops

# Seed
cd <your-repo>
ao quickstart

# Run
/rpi "a small goal"

# Inspect the returned intent, candidate, check evidence, and verdict references

The first session leaves a receipt you can inspect. Later sessions can link prior work or reviewed wiki pages when they are relevant. Whether that trail beats a curated baseline on real tasks is still an open measurement (Devlog #6).

The goal is a repo that remembers, plus gates that refuse "done" without proof. Memory is the cheap habit. Verification is the part that held up when I instrumented it.

This is the engineering edge of a larger project: go to the AI frontier, learn what actually works, then translate it into safe practice for people who aren't engineers. This essay is the engineer-facing half; the translation is the rest.

Related

• 12-Factor AgentOps: The doctrine. DevOps for vibe coding.
• The Knowledge Flywheel: Beads plus an LLM wiki, without automatic-learning claims.
• Building This Website: the discipline applied to a real Next.js build. 48 hours to production, with the bookkeeping to prove it.
• AI Partner: the same discipline, translated for people who don't write code.