Reviewing Agent-Generated Code

Coding agents can produce a diff faster than a person can understand it. That changes the bottleneck in software work.

The difficult part is no longer only generating code. It is deciding whether the change:

• solves the intended problem
• fits the existing system
• preserves behavior outside the happy path
• is safe to merge and operate

Reviewing agent-generated code is therefore not a final glance at a diff. It is a structured process for rebuilding enough understanding to make an informed acceptance decision.

Review Is Different From Approval

An approval prompt asks whether the agent may perform an action, such as running a command or editing a file. A code review asks whether the resulting change is correct.

Control	Question it answers
Tool approval	May the agent run this action?
Sandbox or permission policy	What can the action access or modify?
Verification	Did the change satisfy executable checks?
Code review	Is the implementation correct, maintainable, and appropriate?
Merge approval	May this change enter the shared codebase?

Passing one layer does not satisfy the others. A sandboxed agent can still write incorrect code, and a passing test suite can still miss a security or product requirement.

The Review Pipeline

flowchart LR
    C["Task contract"] --> A["Agent produces<br/>a change"]
    A --> O["Orient<br/>intent and scope"]
    O --> D["Inspect<br/>diff and behavior"]
    D --> V["Verify<br/>tests and evidence"]
    V --> S["Independent<br/>secondary review"]
    S --> J{"Acceptable?"}
    J -->|No| F["Return precise findings"]
    F --> A
    J -->|Yes| M["Merge or release"]

The stages are ordered deliberately. Tests before orientation can tell you that something failed, but not whether the agent solved the right problem. Reading every changed line before understanding the intended behavior can waste time on an implementation that should not exist.

1. Start With A Reviewable Contract

The reviewer needs a stable target. Before implementation, write down:

• the behavior that must change
• behavior that must remain unchanged
• the allowed scope
• architectural or compatibility constraints
• acceptance checks
• actions or files that are explicitly out of scope

A useful contract is concrete enough to reject a plausible but wrong implementation.

Goal:
Reject expired password-reset tokens.

Must preserve:
Existing behavior for valid tokens and already-used tokens.

Constraints:
No database migration. Reuse the repository's clock abstraction.

Acceptance:
Unit tests for valid, expired, and boundary-time tokens.
Existing authentication tests remain green.

The agent’s final summary is not the contract. It is a claim about what the agent believes it did.

2. Orient Before Reading Every Line

First build a map of the change.

1. Read the original task and acceptance criteria.
2. Inspect the repository status and the complete diff against the intended base.
3. List changed, added, deleted, generated, and dependency files.
4. Identify the entry point and the major execution paths affected.
5. Compare the agent’s summary with the actual diff.

Useful questions include:

• Did the agent touch files outside the expected area?
• Did it modify configuration, dependencies, migrations, or generated artifacts?
• Is a large diff mostly mechanical, or does it hide behavioral changes?
• Does the implementation follow nearby repository patterns?

Review the actual repository state, not only a chat transcript or pasted patch. Agents can forget to mention a file, misdescribe a change, or leave untracked output behind.

3. Read In Dependency Order

Alphabetical file order is rarely the best review order. Follow how data and control move through the system.

A practical order is:

1. public contracts, schemas, and types
2. data model and persistence changes
3. core domain or service logic
4. API, command, job, or user-interface entry points
5. tests
6. configuration, deployment, and generated output

For each changed behavior, trace:

input -> validation -> decision -> side effect -> output -> failure handling

Check both what was added and what disappeared. An agent can accidentally remove validation, logging, cleanup, authorization, or a fallback while simplifying the main path.

4. Inspect Behavior, Not Only Style

Agent output often looks polished. Naming, formatting, and explanatory comments can make an implementation feel more complete than it is.

Review the behavioral surface explicitly.

Area	What to inspect
Inputs	Missing, empty, malformed, very large, duplicated, or adversarial values
State	Partial state, stale state, retries, concurrent updates, and idempotency
Errors	Which failures are caught, translated, retried, logged, or hidden
Side effects	Files, database writes, network calls, messages, and external actions
Security	Authorization, secret exposure, injection, unsafe parsing, and trust boundaries
Compatibility	Public API, data format, dependency, platform, and migration effects
Operations	Timeouts, resource use, telemetry, rollback, and supportability

Do not ask only “does this work?” Ask “under which conditions does it stop working, and what happens then?“

5. Demand Deterministic Evidence

Verification should be reproducible by someone who did not watch the agent work.

Run the repository’s real checks, such as:

npm test
npm run lint
npm run build

The exact commands depend on the project. Record:

• the command
• whether it passed
• relevant test counts or artifacts
• checks that could not be run
• any environment assumption needed to reproduce the result

Tests written by the same agent are useful, but they are not independent evidence by themselves. Inspect whether they can fail for the right reason and whether they cover the stated acceptance criteria.

For user-facing behavior, inspect the built application or release artifact. A source-level check cannot prove that bundling, deployment configuration, browser behavior, or installed-package behavior is correct.

6. Use A Second Agent As A Critic, Not An Authority

An independent review agent can search for:

• missed execution paths
• repository convention violations
• security and reliability risks
• insufficient tests
• unnecessary complexity

Give it the task contract, the diff, relevant repository context, and a narrow review criterion. Do not give it the first agent’s persuasive narrative before it has inspected the evidence.

flowchart TB
    D["One code change"] --> C1["Correctness review"]
    D --> C2["Security review"]
    D --> C3["Test coverage review"]
    C1 --> R["Findings with file,<br/>line, impact, and evidence"]
    C2 --> R
    C3 --> R
    R --> H["Human acceptance decision"]

Separate criteria reduce the chance that one broad “review this code” prompt produces shallow comments. They do not make the review independent in the statistical sense. Agents using similar models, context, and assumptions can share the same blind spots.

The human reviewer still owns:

• whether the requirement is correct
• whether the trade-off is acceptable
• whether evidence is sufficient
• whether the change may be merged or released

7. Return Findings That Can Be Acted On

A useful finding contains:

• the affected file and smallest relevant line range
• the observed behavior
• why it matters
• a concrete failure scenario
• the expected behavior
• a verification method

Avoid sending only a replacement implementation. The original agent should understand the failed condition, repair it, and add evidence that prevents recurrence.

After a fix, re-review the changed area and rerun affected regression checks. A repair can invalidate an earlier conclusion.

Checkpoint Strategy

Review frequency should match uncertainty and blast radius.

Strategy	Good fit	Risk
Review every edit	Sensitive code, unfamiliar agent, or exploratory work	High interruption and low throughput
Review at planned checkpoints	Multi-stage work with clear boundaries	A wrong early assumption can survive until the next checkpoint
Review the final pull request	Small, bounded, well-tested changes	Large late rework if scope drifted
Review sampled changes	High-volume, low-risk mechanical work	Unsampled defects may escape

For large work, useful checkpoints include:

• after repository investigation and before coding
• after contract or schema changes
• after the first vertical slice
• before migrations or external side effects
• after tests pass and before merge

Parallel Agents Change The Queue

Running more coding agents increases generation throughput, but reviewer capacity does not automatically increase.

accepted throughput =
min(generation capacity, verification capacity, review capacity, integration capacity)

If five agents create overlapping changes, the team must also resolve dependency order, merge conflicts, duplicated assumptions, and cross-change regressions. Parallelize work only where ownership and interfaces are clear.

Keep each review unit small enough that a reviewer can reconstruct its behavior. A queue of huge diffs is not useful throughput.

Shared Threads As Provenance

A persistent agent thread can preserve:

• the original request
• clarifications and decisions
• commands and tool results
• rejected approaches
• review findings and repair attempts

This helps another engineer understand how the change was produced and learn effective agent workflows from the team.

The thread is supporting evidence, not the source of truth. Link it to the issue or pull request, but keep durable requirements, code, tests, and architectural decisions in the systems that own them.

Before sharing a thread, verify:

• its visibility and retention settings
• whether prompts or tool output contain secrets
• whether customer data or proprietary code was included
• whether links remain accessible to future reviewers

Measuring Review Quality

Lines generated and tasks attempted are weak success metrics. Track outcomes such as:

• time from task start to accepted change
• active reviewer minutes per accepted change
• number and severity of review findings
• repair loops before acceptance
• rollback, incident, or escaped-defect rate
• percentage of checks that are automated and reproducible

Faster generation is useful only when accepted, maintainable changes arrive faster.

Practical Checklist

Before accepting agent-generated code:

• The task contract and acceptance criteria are still valid.
• The complete repository diff matches the intended scope.
• Public contracts, data flow, side effects, and failure paths were traced.
• Added tests fail without the fix and pass with it.
• Existing relevant tests, linting, type checks, and builds pass.
• Security, permissions, dependencies, migrations, and generated files were inspected.
• A secondary review covered the highest-risk criteria.
• Remaining assumptions and checks that could not run are documented.
• The final artifact or user-visible workflow was inspected where relevant.
• The thread, issue, and pull request preserve enough provenance for another reviewer.

Amp’s Current Review Workflow

Amp is one current example of these ideas:

• amp review runs a review agent against a local change
• natural-language review requests can focus on a particular risk
• repository-specific checks can live in .agents/checks/
• separate subagents can evaluate separate review criteria
• web and mobile thread views expose diffs, comments, staging, and change requests

These features can improve review coverage, but they do not replace the task contract, repository checks, or merge authorization. See Amp’s current manual, review announcement, and diff workflow.

Resources

• Amp Code: Next Generation AI Coding - Beyang Liu
• Evaluating Coding Agents
• Verification-Driven Agentic Coding
• Parallel Coding Agents for Large Refactors
• Sandboxing and Permissions for Coding Agents
• Google Engineering Practices: Reviewing a CL