Preventing Agents from Declaring Victory Too Early

Hands-on practice: Project 05. Let the agent verify its own work

You ask an agent to implement a "password reset" feature. It modifies the database schema, writes the API endpoint, adds the email template, runs the unit tests (all pass), and then confidently tells you "it's done." But when you actually try to run it: the password reset link can't be sent because the email service config is missing; the database migration fails halfway through, leaving the schema in an inconsistent state; and the end-to-end flow hasn't been executed even once.

This isn't an isolated incident. The 2017 ICML paper by Guo et al. measured overconfidence in modern neural networks: reported confidence was higher than actual accuracy. It studied image and text classifiers, not coding agents, so read it as an analogy. Agent builders report the same pattern: agents "feel" done well before they are. Your harness must replace the agent's "feelings" with externalized, execution-based verification.

The Slippery Slope

Premature completion declarations almost always follow the same playbook: the code looks okay, syntax is correct, logic seems reasonable, static analysis shows no obvious errors. But the harness doesn't enforce comprehensive execution verification, so the agent skips actually running it or only runs partial tests. It runs unit tests but skips integration tests; it runs tests but doesn't check coverage. In the end, "the code looks fine" is taken as evidence that "the feature is complete."

Information is lost at every step. From task specification to code implementation to runtime behavior, every transformation can introduce bias, and every skipped verification compounds the information asymmetry.

Three-Layer Termination Check

The Rule and Its Condition

Run all three termination layers, gated in order, before declaring anything done that touches a code path the running app actually executes, which is nearly every change. Stop at layer 1 only when nothing in the diff can run: a comment, a doc file, a config value nothing reads yet. The moment a change can execute, skipping layer 2 or layer 3 is the exact premature-victory pattern this lecture opened with.

The condition is observable in the diff itself: does any changed file get imported, called, or read by a running process? If yes, all three layers gate the completion claim. If every changed file is inert, prose or comments no code path consumes, layer 1 is the whole check.

Core Concepts

• Premature Completion Declaration: The agent asserts the task is complete, but unmet correctness specifications remain. The core problem is that the agent judges based on local, code-level confidence, while system-level correctness requires global verification.
• Confidence Calibration Bias: A systematic gap between the agent's self-reported completion confidence and the actual completion quality. For complex multi-file tasks, this bias is significantly positive: the agent is consistently more confident than its actual performance warrants.
• Termination Criteria: A clear, executable set of judgment conditions defined in the harness. The agent must satisfy all conditions before declaring completion. "Done" shifts from a judgment call to a check that passes or fails.
• Verification-Validation Dual Gate: The first layer (verification) checks whether the code correctly implements the specified behavior; the second layer (validation) checks whether system-level behavior meets end-to-end requirements. Both must pass before the task is considered complete.
• Runtime Feedback Signals: Logs, process states, and health checks from program execution: these form the objective basis for the harness to judge completion quality.
• Completion Priority Constraint: First verify functional correctness, then address performance, and finally handle style. No refactoring is allowed until core functionality has been verified.

Passing Unit Tests ≠ Task Complete

This is the most common trap, and the most dangerous one. The agent writes code, runs unit tests, sees all greens, and says "done." But the design philosophy of unit tests (isolating the tested unit and mocking dependencies) is precisely what makes them incapable of detecting cross-component issues:

Interface Mismatch: The renderer passes a relative file path to the preload script, but the preload script expects an absolute path. Their respective unit tests both use mocks and both pass. The issue only surfaces during end-to-end testing.

State Propagation Errors: A database migration changes the table schema, but the ORM's caching layer still holds cache entries for the old schema. Unit tests run in a fresh mock environment every time, so this kind of cross-layer state inconsistency never surfaces.

Environment Dependency: Code behaves correctly in the test environment (where everything is mocked) but fails in the real environment due to configuration differences, network latency, or service unavailability.

"Refactoring While We're at It" is Poison to Completion Judgment

Claude Code has a common behavioral pattern: it starts refactoring code, optimizing performance, and improving style before the core functionality has passed verification. Knuth's adage that "premature optimization is the root of all evil" takes on new meaning in the agent scenario: refactoring shifts the boundary between verified and unverified code, potentially breaking code paths that were previously implicitly correct.

Systematic Bias in Self-Evaluation

Anthropic discovered a deeper failure pattern in their 2026 research: when an agent is asked to evaluate its own work, it systematically provides overly positive assessments, even when a human observer would judge the quality as clearly substandard.

This problem is especially severe on subjective tasks (such as design aesthetics). Whether a "layout is polished" is a judgment call, and the agent reliably skews positive. Even on tasks with verifiable outcomes, the agent's poor judgment degrades its performance.

Asking the agent to be "more objective" doesn't fix it: the same model both generates and evaluates, and tends to be generous with itself.

Separate the "worker" from the "checker."

An independent evaluation agent, specifically tuned to be "nitpicky," is far more effective than having the generating agent evaluate itself. Anthropic's experimental data:

This is the exact same model (Opus 4.5) with the exact same prompt ("build a 2D retro game editor"). The only difference is the harness: from "running bare" to "planner expands requirements → generator implements feature by feature → evaluator performs actual click testing using Playwright."

Source: Anthropic: Harness design for long-running application development

How to Prevent Premature Completion Declarations

1. Externalize Termination Judgment

The completion judgment should not be made by the agent itself. The harness independently executes termination validation, using runtime signals as input rather than the agent's confidence. Write it in harness/definition-of-done.md and reference that file from AGENTS.md so every session loads it:

2. Build a Three-Layer Termination Validation

• Layer 1: Syntax and Static Analysis. Lowest cost, least information, but must pass. This is the bare minimum: you need to spell the words correctly before reading further.
• Layer 2: Runtime Behavior Verification. Test execution, application startup checks, critical path validation. This is the core evidence of completion: not just written, but runnable.
• Layer 3: System-Level Confirmation. End-to-end testing, integration validation, user scenario simulation. The last line of defense against premature declarations: not just runnable, but correct.

3. Provide Actionable Error Feedback to the Agent

OpenAI introduced a particularly effective pattern in their Codex practice: error messages written for agents should include repair instructions. Don't just tell the agent "it's wrong." Point out exactly what's wrong and how to fix it. Don't use "Test failed", use "Test failed: POST /api/reset-password returned 500. Check that the email service config exists in environment variables. The template file should be at templates/reset-email.html." This kind of specific, actionable feedback lets the agent self-correct without human intervention.

4. Capture Runtime Signals

Effective runtime signals include:

• Did the application successfully start and reach a ready state?
• Did critical feature paths execute successfully at runtime?
• Were database writes, file operations, and other side effects correct?
• Were temporary resources cleaned up?

Trace It: Password Reset, "Done"

Task: implement password reset, database operations, email sending, API endpoint. The agent modifies the schema, writes the endpoint, adds the email template, runs unit tests (all pass), and declares completion.

Predict before reading on: using only the three-layer model above, what has this agent actually verified, and what hasn't it?

Trace it: unit tests passing is layer-2 evidence at best (isolated behavior, mocks included); lint or typecheck covers layer 1. Nothing here executed layer 3, the running system. So three things stay unverified: the reset link's actual send-and-verify flow, whether the database migration completed cleanly, whether the email service is even configured in the target environment.

Enforce termination validation instead: start the full app, run the complete reset flow, check database state, and those three gaps surface immediately, while the context that produced them is still loaded, instead of as a production bug report days later.

Faded case

Skip to the open case if this is familiar. A different session: the agent adds a "bulk CSV export" feature to an admin dashboard. It writes the export function, wires a "Download CSV" button, runs the existing unit test suite, which mocks the file-write call, and every test stays green. The agent reports the feature complete. You open the app, click the button once, and a file downloads with the right column headers. Before you accept "done," work out which of the three termination layers that single click satisfies and what remains unverified.

Two gaps from earlier apply. Mocked writes hide real file-system behavior. And one click through a happy path is not a system-level flow exercised end to end, edge cases included.

Open case

Pick a feature in your own repository that got marked "done" recently, yours or an agent's, ideally one you did not personally verify at the time, so you are not just recalling the answer. Read the diff cold. Decide, using only the three layers above, what actually ran: which parts were exercised by a real process, which were only asserted by a mocked test, and which were never run at all, just written and eyeballed. Name the specific layer whose evidence is missing, the way the faded case named the mocked write and the untested edge case.

Exercise 1 below turns this into a repeatable termination-validation design. If nothing in your repo is disputable enough to try this on, the next feature you or an agent finishes is the open case: run the check before you accept the claim, not after a bug report reopens it.

Key Takeaways

• Agents are systematically overconfident: written code is not verified code.
• Completion judgment must be externalized: the harness verifies independently. Don't trust the agent's "feelings."
• All three layers of validation are essential: syntax passes, behavior passes, system passes, layer by layer, no shortcuts.
• Error messages should include specific repair steps, enabling the agent to self-correct rather than just telling it "it's wrong."
• No refactoring until core functionality is verified: the completion priority constraint is the key to preventing premature optimization.

Example Files

Copy-ready course examples referenced in this lecture.

clean-state-checklist.md

Four conditions a session must leave behind before "done" is allowed, startup works, progress recorded, no undocumented half-finished steps, next agent can run the standard verification path.

victory-detector.ts

Runnable claimed-vs-actual auditor: three tasks the agent marked complete, each check recording what the agent said against what's really true, the gaps (skipped tests, forgotten auth, leftover old code) are the premature-victory signature made visible.

/** * victory-detector.ts * * Simulates an agent that claims task completion, then checks the claimed * state against actual verification. Outputs: claimed vs actual, highlighting * gaps between what the agent said and what's really true. */// ---------------------------------------------------------------------------// Types// ---------------------------------------------------------------------------interface Task {  name: string;  claimedComplete: boolean;  checks: VerificationCheck[];}interface VerificationCheck {  description: string;  claimed: boolean; // What the agent says  actual: boolean;  // What is actually true  severity: "critical" | "warning";}// ---------------------------------------------------------------------------// Simulated tasks with claimed vs actual states// ---------------------------------------------------------------------------const tasks: Task[] = [  {    name: "Add search endpoint",    claimedComplete: true,    checks: [      {        description: "Route handler exists",        claimed: true,        actual: true,        severity: "critical",      },      {        description: "Authentication middleware applied",        claimed: true,        actual: false, // Agent forgot auth        severity: "critical",      },      {        description: "Input validation added",        claimed: true,        actual: true,        severity: "critical",      },      {        description: "Unit tests written",        claimed: true,        actual: false, // Agent skipped tests        severity: "critical",      },      {        description: "Integration tests pass",        claimed: true,        actual: false, // No tests to run        severity: "critical",      },      {        description: "Documentation updated",        claimed: true,        actual: false, // Agent skipped docs        severity: "warning",      },    ],  },  {    name: "Fix pagination bug",    claimedComplete: true,    checks: [      {        description: "Bug reproduction case confirmed",        claimed: true,        actual: true,        severity: "critical",      },      {        description: "Root cause identified",        claimed: true,        actual: true,        severity: "critical",      },      {        description: "Fix applied",        claimed: true,        actual: true,        severity: "critical",      },      {        description: "Edge cases tested",        claimed: true,        actual: false, // Only happy path tested        severity: "warning",      },      {        description: "Regression tests added",        claimed: true,        actual: false,        severity: "critical",      },    ],  },  {    name: "Refactor auth module",    claimedComplete: true,    checks: [      {        description: "Old code removed",        claimed: true,        actual: false, // Old code still present        severity: "warning",      },    ],  },];for (const task of tasks) {  const gaps = task.checks.filter((c) => c.claimed && !c.actual);  const critical = gaps.filter((c) => c.severity === "critical").length;  console.log(`${task.name}: claimed complete=${task.claimedComplete}, gaps=${gaps.length} (critical=${critical})`);  for (const gap of gaps) {    console.log(`  - ${gap.description}`);  }}

What You Have Now

A definition-of-done the agent must satisfy with command output, not prose.

Write it as checks that produce output the agent must show, commands run, gates passed, so "it works" claims are backed by evidence instead of confidence. If done can be asserted without running anything, tighten it.

Save it as harness/definition-of-done.md. The lab's checker grades that file for a Definition of Done heading, the line Do not proceed to level 2 if level 1 fails, and no unfilled placeholders.

Further Reading

• On Calibration of Modern Neural Networks - Guo et al., Proves modern deep networks are systematically overconfident
• Building Effective Agents - Anthropic, The critical role of runtime evidence in completion judgment
• Harness Engineering - OpenAI, Premature completion declaration is one of the main failure modes of agents
• The Art of Software Testing - Myers, Classic reference on testing method hierarchies and effectiveness

Exercises

1. Termination Validation Function Design: Design a complete termination validation for a task involving database migration and API modifications. List the required runtime signals and the pass/fail criteria for each. Run it on a real task and record which hidden issues it discovers.

2. Calibration Bias Measurement: Select 10 coding tasks of different types. Record the agent's self-reported completion confidence versus the actual completion quality. Calculate the bias and analyze its relationship with task complexity.

3. Multi-Layer Defense Experiment: Run three configurations on the same set of tasks: (a) static analysis only, (b) add unit testing, (c) full three-layer validation. Compare the proportion of premature completion declarations and the number of uncaught defects.

Check your understanding