Verification-Driven Agentic Coding

Agentic coding works best when the agent has a target it can check by itself.

Without a verification signal, the agent stops when the code looks plausible. With a signal, it can run the check, read the failure, change the code, and repeat until the output is actually better.

Why Verification Changes Agent Performance

Many software tasks are difficult to solve but comparatively cheap to check.

A model may need several attempts to repair a type mismatch across unfamiliar modules. A compiler can reject each invalid attempt in seconds. The compiler does not make the model smarter, but it gives the agent an objective boundary and a useful next observation.

flowchart LR
    G["Generate a candidate"] --> V["Run an external check"]
    V --> R{"Accepted?"}
    R -->|No| E["Return a specific failure"]
    E --> G
    R -->|Yes| W["Run wider verification<br/>and human review"]

This works best where the check is reliable and the failure is informative. It works poorly when the test is flaky, the build takes too long, or success measures only a weak proxy for the required behavior.

Verification reduces uncertainty. It does not prove that arbitrary software is fully correct.

The Core Pattern

Ask the agent to inspect the relevant code path.
Define the smallest check that proves the behavior.
Write or identify that check before changing the implementation.
Let the agent implement the fix.
Make the agent run the check and show evidence.

For code, the check might be a failing unit test. For UI, it might be a screenshot comparison. For data work, it might be a fixture diff. For infrastructure, it might be a dry-run plan.

Good Verification Signals

Signal	Best for	Example
Unit test	Pure functions, edge cases, regressions	`npm test -- auth-refresh.test.ts`
Integration test	API, database, service boundaries	`pytest tests/api/test_login.py`
Build/typecheck	Compile-time and contract errors	`npm run build`, `tsc --noEmit`
Lint/format	Mechanical consistency	`npm run lint`
Screenshot	Visual UI changes	Browser or Playwright screenshot after the edit
Fixture diff	Parsers, serializers, generated output	Compare generated JSON against expected output
Log replay	Debugging production-like failures	Pipe an error log into the task context
Dry run	Cloud, migration, deployment work	`terraform plan`, migration preview, `--dry-run`

Properties Of A Useful Verification Signal

Property	Meaning	Warning sign
Objective	The result follows from executable rules or observable evidence	Success depends only on the agent saying the result looks right
Fast	The smallest relevant check returns quickly	Every edit waits for a complete CI pipeline
Low-noise	The same state normally produces the same result	Unrelated tests fail intermittently
Diagnostic	Failure output identifies what condition was violated	The check reports only `failed` or a large undifferentiated log
Representative	The check exercises behavior users or dependent systems rely on	The test passes while the public workflow remains broken
Difficult to game	Passing requires satisfying the intended condition	The agent can delete an assertion, update a snapshot, or silence the error
Graduated	Partial or layered feedback helps identify progress	The only available result is one late pass/fail gate

A check does not need every property to be useful. The table helps explain why two tests with the same pass/fail interface can provide very different agent performance.

Layer Checks By Cost

Run the cheapest relevant signal first.

flowchart TB
    C["Changed code"] --> F["Format"]
    F --> L["Lint and type check"]
    L --> U["Focused unit test"]
    U --> I["Relevant integration test"]
    I --> B["Build and wider suite"]
    B --> S["Staging, visual,<br/>or production evidence"]

This ordering shortens the repair loop:

formatting catches mechanical drift
linting and types catch local contract errors
a focused test checks the changed behavior
integration checks expose boundary effects
wider checks detect regressions
staging and runtime evidence test the deployed system

Not every task needs every layer. A documentation change may need only a docs build and rendered-page inspection. A database migration may need schema validation, migration tests, a dry run, rollback evidence, and staging observation.

Feedback Latency Is Part Of The Agent Environment

An agent that receives a useful failure in ten seconds can make several informed attempts while another agent is still waiting for CI.

Track:

time to first useful failure
p50 and p95 duration for focused checks
time spent waiting for unavailable environments
percentage of checks that can run locally
number of retries caused by infrastructure rather than code

Do not optimize by deleting meaningful checks. Split large suites, improve fixtures, cache safe dependencies, and expose smaller commands that preserve the same guarantees.

Flaky Checks Corrupt The Loop

A human may recognize a known flaky test and rerun it. An agent can interpret the same noise as evidence that its correct change is wrong.

It may then:

modify unrelated code
weaken or remove the test
consume repeated tokens and CI runs
stop without knowing which result to trust

Quarantine a flaky check only with an owner, visibility, and a repair plan. Measure flakiness as an agent-readiness problem, not only a CI inconvenience.

TDD Prompt

Investigate this bug first. Do not change implementation yet.

Write the smallest failing test that reproduces the reported behavior.
Run only that test and confirm it fails for the expected reason.
Then implement the fix, rerun the test, and show the final command output.

This works because the agent gets a concrete loop:

flowchart LR
    A["Inspect code"] --> B["Create failing check"]
    B --> C["Implement change"]
    C --> D["Run check"]
    D --> E{"Passes?"}
    E -->|No| C
    E -->|Yes| F["Return evidence"]

UI Prompt

Use the existing app locally.

Reproduce the UI issue, take a screenshot, apply the smallest fix,
take another screenshot, and compare the result against the expected state.
Do not call the task done unless the visual difference is resolved.

For UI work, a text description is often not enough. Screenshots give the agent a readable result, especially when spacing, overflow, contrast, and layout are involved.

Autonomous Browser Testing

A generated interface can look complete while important behavior behind it is missing.

Examples include:

a button with no working event handler
a form that reports success without saving data
a dashboard that displays mock values instead of an API response
a reset-password screen that never sends an email
a checkout page that updates visually but never creates an order

These are sometimes called painted doors: the entrance is visible, but it does not lead anywhere.

Static analysis, unit tests, and a successful build may all pass because each checks only part of the system. Browser testing adds evidence at the level where the user encounters the product.

Test journeys, not random clicks

A useful browser check begins with a user journey and an observable result:

Journey: create a customer-feedback entry

Given:
- the application is running with an empty test database

Actions:
- open the feedback page
- enter a name and message
- submit the form

Expected evidence:
- a success state is visible
- the network request succeeds
- one matching record exists in the test database
- the entry appears after the page is refreshed

The visible success message is not enough. The database record alone is not enough. Together, they establish that the interface, API, and persistence path agree.

Combine several evidence channels

Evidence	What it can reveal	What it may miss
Accessibility tree or DOM	Controls, labels, roles, and visible state	Backend side effects
Browser actions	Whether a user can complete the interaction	Hidden data corruption
Network requests	Endpoint, status, payload, and timing	Incorrect downstream persistence
Test database state	Whether data was saved correctly	Broken presentation or navigation
Console and service logs	Runtime errors and failed integrations	Silent business-rule errors
Screenshot	Layout, overflow, contrast, and visual result	Invisible semantics and side effects
Trace	Sequence of actions, requests, logs, and page states	Requirements omitted from the scenario

No single channel proves the whole application. Select evidence based on the behavior being changed.

flowchart LR
    J["Critical user journey"] --> B["Browser interaction"]
    B --> U["Visible UI state"]
    B --> N["Network and API evidence"]
    B --> D["Database or durable state"]
    B --> L["Console and service logs"]
    U --> A["Evaluate acceptance criteria"]
    N --> A
    D --> A
    L --> A
    A -->|"Fail"| F["Return focused repair evidence"]
    A -->|"Pass"| R["Save a reusable regression check"]

Generic browser tools versus executable tests

An agent can interact with a browser through generic tools such as:

open a page
click an element
fill an input
inspect accessibility information
take a screenshot

This is useful for exploration and one-off diagnosis.

The agent can also generate an executable Playwright test:

import { expect, test } from '@playwright/test';

test('submitting feedback persists the entry', async ({ page }) => {
  await page.goto('/feedback');
  await page.getByLabel('Name').fill('Asha');
  await page.getByLabel('Message').fill('Please add CSV export.');
  await page.getByRole('button', { name: 'Send feedback' }).click();

  await expect(page.getByText('Thanks for your feedback')).toBeVisible();

  await page.reload();
  await expect(page.getByText('Please add CSV export.')).toBeVisible();
});

Executable tests offer several advantages:

they use a general programming language rather than a fixed catalog of browser actions
assertions make the expected behavior explicit
the exact scenario can be rerun after a repair
the test can enter the normal regression suite
traces and screenshots can be retained for later debugging

Generic tools remain useful when the agent is discovering the workflow or handling an interaction that has not yet been encoded. A practical system may explore with browser tools, then preserve important behavior as a focused test.

Generated tests still need review

A test written by the same agent that wrote the feature can reproduce the agent’s misunderstanding.

Inspect whether the test:

starts from a meaningful state
exercises the public user journey
checks the important side effect
would fail if the implementation were removed or replaced with a fake
uses resilient roles and labels instead of brittle layout selectors
avoids weakening an existing assertion
cleans up test data

For high-risk behavior, derive acceptance scenarios from independently reviewed requirements and run the test in a clean environment.

Keep browser noise out of the coding context

Browser runs can produce large accessibility snapshots, screenshots, network traces, and logs. A dedicated verification subagent can perform the journey and return only:

the failed acceptance criterion
the first useful error
references to retained traces or screenshots
a minimal reproduction
whether the failure is in the UI, API, data, or environment

This preserves the evidence without flooding the main implementation context. See Context Engineering for Long-Running Agents.

Define human-takeover boundaries

Some browser steps should not be automated around:

CAPTCHA
account consent
hardware security confirmation
personal login credentials
regulated approval
destructive production actions

Pause with an explicit takeover request or move the scenario to a safe test environment. Do not ask the model to bypass the control.

Current Replit documentation uses this pattern for App Testing: the agent can test and attempt corrections in a browser, but it requests takeover for interactions such as account login. This is one product example, not proof that every workflow can be verified autonomously.

Evidence To Ask For

Ask for evidence, not just a claim:

the exact command it ran
the relevant passing test output
before/after screenshots for visual work
the changed files
any checks it could not run and why

What Automated Checks Do Not Prove

Passing checks do not establish that:

the requirement itself was correct
important behavior was included in the suite
the implementation is maintainable
authorization and privacy boundaries are appropriate
a user-facing design is understandable or accessible
the change will behave correctly under every production condition

Tests written by the implementation agent are also not independent evidence by themselves. Inspect whether they fail without the fix, exercise public behavior, use representative data, and reject plausible wrong implementations.

Use deterministic checks first, then add human review, security analysis, visual inspection, and production evidence according to risk.

Common Failure Modes

Failure	What to do instead
Agent updates snapshots without explaining why	Ask it to prove the new output is correct
Agent runs the whole suite too early	Start with the smallest relevant test
Agent claims success without output	Require command output or screenshot evidence
Agent suppresses an error	Tell it to fix root cause, not silence the check
No runnable tests exist	Start with a smoke script, fixture, or manual checklist
Test passes only because the agent weakened it	Review the assertion and reproduce the original failure
Flaky failure sends the agent into unrelated edits	Repair or isolate the flaky check with explicit ownership
Local checks and CI use different commands or versions	Make the environment and entry points reproducible

When To Skip This

Do not force a full TDD loop for small documentation edits, typo fixes, or obvious one-line changes. Use the cheapest meaningful check: build the docs, run markdown lint, or read the changed page.