Harness Design Patterns for Long-Running Agents

Long tasks often fail because a new session cannot reconstruct what the previous session changed. Context compaction helps preserve conversation history, but it is not a substitute for project state, acceptance criteria, or recoverable checkpoints. A harness lets every session resume quickly and deliver one verified increment.

---

Separate the model, agent loop, and harness

User outcome
  ↓
Harness: startup, state, budget, access, retries, handoff, evaluation
  ↓
Agent loop: observe → plan → call tools → record result
  ↓
Model: reason and generate within the current context
  ↓
Environment: repository, browser, database, and external APIs

---

Four failure modes a harness should prevent

Anthropic's long-running agent experiment used distinct initializer and coding-agent prompts. Those are roles, not necessarily separate services or different models.

---

A minimal project layout

agent-harness/
├── init.sh
├── feature_list.json
├── progress.md
├── runs/
│   └── 2026-08-25T10-00-00Z.json
└── workspace/

Make the feature list the acceptance source of truth

[
	{
		"id": "checkout-card-payment",
		"priority": 1,
		"description": "A signed-in user can complete a card payment",
		"steps": [
			"Open checkout with one item",
			"Enter valid test card details",
			"Submit payment",
			"Verify order status is paid",
			"Verify receipt remains visible after refresh"
		],
		"passes": false,
		"evidence": []
	}
]

The coding session may change passes from false to true only after appending evidence. It must not delete a test, weaken an assertion, or rewrite the user outcome merely to obtain a pass.

---

The initializer session builds the runway

The initializer should not implement ten features. Its job is to create a reproducible working environment:

1. Break the outcome into end-to-end features.
2. Write an init.sh that starts the environment deterministically.
3. Define progress and run-record formats.
4. Run one smoke test to prove that the baseline works.
5. Create the first clean checkpoint.

#!/usr/bin/env bash
set -euo pipefail

test -f package.json
npm ci
npm run build
npm run dev

A production init script also checks ports, health endpoints, timeouts, and log locations. Do not use a fixed sleep as proof that a service is ready.

---

Every coding session follows the same protocol

1. Confirm the working directory
2. Read progress.md, feature_list.json, and recent Git history
3. Start the environment and run the smoke test
4. Select the highest-priority failing feature
5. Implement the smallest vertical slice
6. Run unit, integration, and actual UI checks
7. Save evidence, then set passes=true
8. Update progress and create a clean checkpoint

progress.md should answer operational questions, not tell a story:

## Current state

-   Active feature: checkout-card-payment
-   Last verified commit: 1a2b3c4
-   Dev command: npm run dev
-   Smoke test: passed at 2026-08-25T10:22:00+10:00

## Evidence

-   Unit: payment.service.test.ts
-   E2E: runs/checkout-card-payment.webm
-   Database: order ord_123 has status paid

## Next safe action

-   Verify the receipt survives a hard refresh

---

Evidence has levels

If the outcome is “publish a video,” a local file or successful upload is not the finish line. The harness should require the public provider URL and status read-back.

---

Recovery and idempotency

Give each task a stable ID and a small state machine such as pending, running, and verified. After a crash:

1. Read the latest append-only run record.
2. Check whether the external side effect already happened.
3. If it happened, attach evidence instead of repeating it.
4. If it did not happen, resume from the latest checkpoint.
5. If the state cannot be determined, record unknown; do not guess.

Payments, email, publishing, and deletion require idempotency keys or provider request IDs.

---

Budgets and stop conditions

Before execution, define:

• Maximum model calls and token budget.
• Tool-call and session timeouts.
• A threshold for repeated identical errors.
• Operations that need human approval.
• Conditions for blocked and permitted fallbacks.

A long-running task without stop conditions is an uncontrolled loop, not a dependable autonomous agent.

Practice task

Design a harness for “upload a CSV and generate an analysis report”:

1. Write at least 12 feature cases, including empty files, invalid encodings, and duplicate uploads.
2. Add the init script and health check.
3. Limit each session to one feature.
4. Preserve both the database record and the downloadable file as evidence.
5. Simulate a crash after writing the file but before updating the database, then verify recovery creates no duplicate report.

Self-check

• A new session does not need to guess what happened previously.
• The coding agent cannot delete or weaken feature criteria.
• Passing includes evidence from the user flow.
• External side effects can be recovered through idempotency.
• The harness has explicit budgets, timeouts, and approval boundaries.

Related reading

• Claude Code Context Management
• Agent Evaluation
• Claude Code Sandboxing

Official references

• https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents
• https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents