---
title: A CLI Agent Harness Playbook
date: 2026-02-03
summary: The terminal loop I hold Claude Code and Codex to, the skills that encode it, the repo files that keep two assistants reading one source of truth, and the failure patterns that shaped all of it.
status: published
tags: claude code, codex, cli, developer workflow, ai engineering, skills, testing
mediaSourcePath: /ai-lab/
mediaSourceSelector: .projects-grid, .hero, main
mediaSourceTheme: dark
imageAlt: The AI Lab hub, built almost entirely through the terminal loop this article describes.
imageCaption: Inspect, patch, test, hand off. The discipline is the product.
---

# A CLI Agent Harness Playbook

When I need a change I'd stake a deploy on, I stay in the terminal. Inspect, patch, test, summarize. It's the tightest feedback loop I've found for AI-assisted work, and it's the one that produces diffs I'm willing to merge without re-reading every line.

This is how I run Claude Code and Codex day to day, and what I've written into the repos so the discipline doesn't depend on my memory.

## The contract for every run

Five steps. When I skip one, I usually pay for it later.

1. **Scope, with non-goals.** Writing down what I'm not doing matters as much as what I am. "Fix the rate limiter's memory growth. Do not touch the Firestore-backed path."
2. **Read the minimum.** Broad reads produce broad edits. If the task is one function, the assistant reads that function and its callers, not the directory.
3. **One behavior change per patch.** Bundled changes hide the one that broke something.
4. **Targeted tests first, then wider.** The spec that covers the change, then the canary, then the full batch only when the change earns it.
5. **A handoff note.** Files changed, what was verified, what wasn't. Without this the run isn't finished, it's just stopped.

I turned the parts of that loop that kept getting skipped into Claude Code skills. `/diagnose` forces a written, ranked hypothesis list before any fix. `/implement-plan` picks up an existing plan document and executes it phase by phase instead of writing a new plan. `/audit` runs a CSS, JavaScript, and HTML checklist over the diff before a commit. Those aren't clever prompts. They're the steps I noticed myself rationalizing away at the end of a long session.

## Routing between Codex and Claude Code

I don't have a benchmark-driven rule for this. I have a task-shape rule.

| Task | Where it starts | When I switch |
| --- | --- | --- |
| Small fix plus its test | Codex | The root cause turns out to be architectural |
| Multi-file refactor | Claude Code, plan first | Execution is the bottleneck, not understanding |
| Release readiness review | Claude Code | Findings are tooling-heavy and need patching |
| Script and tooling upkeep | Codex | The policy question is ambiguous |

Escalation is a response to a concrete failure, not a reflex. If the first choice is working, I don't upgrade for the sake of it.

## One source of truth for two assistants

When more than one assistant touches the same repo, they have to read the same rules. The pattern I've settled on is that the harness-specific files are adapters, not forks of the policy.

On [good-vibes](https://github.com/benmcnulty/good-vibes) that looks like `AGENTS.md` for Codex, `CLAUDE.md` for Claude Code, and `.github/copilot-instructions.md` for Copilot, each stating that assistant's role and pointing at the shared roadmap in `docs/`. On this site, `CLAUDE.md` is the canonical file and `AGENTS.md` is deliberately short: it tells any other agent to go read `CLAUDE.md`.

I review changes to those files as carefully as I review code. Small drift between them is how you end up with one assistant running the full Playwright suite in a single process while the other one batches it, and then arguing with the results.

## Validating paid APIs without burning credit

The chat on this site depends on OpenRouter, and before a functions deploy I want to know the model chain is alive. The gate is deliberately cheap:

```json
{
  "messages": [{ "role": "user", "content": "Reply with: ok" }],
  "promptKey": "aiLab",
  "model": "<one model from the chain>"
}
```

One probe per model. No long prompts in a health check. No blind retries. Every failure reason logged. The predeploy spec reads the model list from the same config file the function uses. A gate with its own hard-coded model IDs is a gate that goes stale the day a provider retires one, and free-tier catalogs retire models without asking.

## Local models in the loop

Bounded tasks are where local models earn their place: offline scaffolding, low-risk text transforms, a first draft of documentation. [Local Crew](https://github.com/benmcnulty/localcrew) is the terminal-first version of that idea taken all the way. It pools Ollama and OpenAI-compatible endpoints across the machines on my network, routes tasks by resource tier, and runs the same inspect, patch, verify loop across devices instead of on one.

I still don't let a local model be the final authority on a high-risk multi-file change. But "final authority" is a small fraction of the work.

## Failure patterns that shaped the playbook

**Scope creep in autonomous runs.** The fix was a changed-file check before the test suite runs. If the diff is wider than the task, stop and ask.

**Green tests, wrong behavior.** Assertions that were too loose. The fix was tying tests to the acceptance criteria in the plan, not to whatever the code happened to do.

**Strong reasoning, weak execution.** Splitting planning and patching into separate runs with a checkpoint between them. Let each tool do the part it's good at.

**Silent skips.** The one I watch for most. A runner that treats a killed process as a pass, or a suite that skips itself when a dependency is slow to start, looks green right up until it matters. The fix is a runner that's honest about exit codes and a readiness probe that retries before it gives up.

## What holds up

The advantage of a CLI harness isn't the model inside it. It's that the loop is auditable. Every step leaves something a reviewer can read: a plan, a diff, a test result, a note. As models improve, the teams that enforce that contract will compound the gains. The teams chasing leaderboards will keep re-reading diffs.