Once AI-assisted coding becomes common, AI-assisted code review feels like the obvious next step.

But after you actually run it for a while, the hard question is not whether the model can understand a diff. It can, especially for boundary conditions, exception handling, missing tests, interface compatibility, and obvious logic errors. The first screening pass is often valuable. The real problem is different: how do you turn AI Review into a stable, controllable engineering process with predictable cost?

If every review asks a general-purpose Agent to decide on the fly how to fetch code, read the PR, comment, and approve, the first version looks flexible. After a while, it becomes another source of uncertainty.

This article records the process of changing Codex Review from “it can run” into “it is a service.”

Start by Looking at Existing Patterns

There are already many AI Review practices on GitHub. A representative example is Google’s run-gemini-cli GitHub Action. It can call Gemini CLI inside a GitHub workflow to review PRs, triage issues, and respond to manual triggers from PR comments.

That shape is natural: a developer opens a PR, the Action reads the diff and repository context, generates review comments, and provides suggestions when needed. To the user, it feels like a code-review assistant living inside GitHub.

The strengths are clear:

• The entry point is GitHub, where developers already work.
• Automatic and manual triggers both feel natural.
• Review results stay inside the PR for discussion.
• The team does not need to maintain much infrastructure.

But there is also a practical limitation: strategy, quotas, and cost of external services are not fully under your control. A free quota available today may become a limited trial tomorrow. Pricing that works for personal projects may need to be recalculated for daily team usage.

If code review has become part of the team’s daily workflow, it should not remain merely an external tool call. It needs to become an internal, controllable service.

The Goal Is Not Simply Replacing Gemini With Codex

It is tempting to frame the requirement as: we used Gemini Review before, now use Codex Review instead.

That is only half right.

The model is only one component. What really needs to be built is a review workflow:

• When should review be triggered?
• Who can trigger it?
• Which PR is reviewed?
• Should approval happen by default?
• Where should comments be written?
• How should failures be reported?
• Where can history be inspected?
• How should cost be counted?
• What happens when the same PR is reviewed repeatedly?
• Can different repositories use different rules?

These questions should not be left for the model to improvise each time. The model should read code and discover issues. It should not organize the whole engineering workflow.

So the goal becomes: use Codex for code understanding, and use deterministic service code for orchestration.

Version One: A Developer Machine Polling PRs

The earliest version was direct: find a developer machine, install Codex and GitHub CLI, write a scheduled task that scans open PRs in target repositories every minute.

If a PR has not yet been approved, trigger Codex Review automatically and post the result back to GitHub.

The biggest advantage was speed.

No GitHub integration changes, no web page, no chat interaction. Once a PR was opened, the background job would eventually find it. To developers, it felt like an automatic reviewer had joined the team.

But after some time, the problems were obvious.

First, token usage was not controlled.

The scheduled task only knew that a PR was open and not approved. It did not know whether the PR truly needed AI Review. Small edits, temporary PRs, draft PRs, and repeated pushes all entered the scan range. The model had to read context, read diffs, and generate conclusions. All of that costs tokens.

Second, users had no way to express intent.

Developers could not easily say:

• Do not review this PR yet.
• Focus on compatibility.
• Only check tests.
• Re-review after these changes.

All policy lived inside the scheduled task.

Third, state was invisible.

Where is the job now? Is GitHub authentication expired? Did repository clone fail? Did Codex time out? Was the comment posted? Did approval succeed?

These questions required checking the developer machine or GitHub manually. Ordinary users could not easily understand the status.

The first version proved that Codex could review PRs. It also proved that full-repository polling was not a good long-term form.

Version Two: Trigger From Chat

The second version moved the entry point into chat.

The idea was simple: a developer sends a PR URL in chat and asks the bot to review it. The bot understands the message, parses the PR link, and calls Codex Review.

This was better than polling. Users could trigger review intentionally instead of waiting for background scanning. They could also see execution results in the chat window, lowering the entry barrier compared with command-line usage.

But this version exposed another problem quickly: the chain was too long.

A review request first went through the chat bot. The bot had to understand natural language, decide which capability to call, and convert parameters into a concrete command. Users might write:

• Take a look at this PR.
• Review this link.
• Please approve it.
• Run Codex on this.
• Check whether this PR has problems.

These are natural expressions, but they are not a stable protocol for the program behind the bot. If the bot assembles the command incorrectly, uses the wrong parameter name, or mismatches a field, the first execution fails. If another Agent then tries to fix that failure, the chain becomes even less controllable.

In practice, several failure modes appear:

• Review command parameters are assembled incorrectly.
• Skill or tool field names differ from the real interface.
• GitHub clone or authentication errors are relayed through several layers, making the root cause unclear.
• Streaming replies feel slow, as if the result is typed character by character.
• It becomes hard to verify whether the final action came from the intended review capability or from another tool chosen mid-flight by the Agent.
• GitHub comment formatting varies; suggestion blocks may look messy.

This shows a key point: code review is a highly deterministic engineering task. The core process should not be fully organized by a general-purpose Agent at runtime.

An Agent can understand user intent and code. But fetching PRs, running commands, formatting comments, approving, recording history, and recovering from failures should be ordinary program logic.

Version Three: Make It an Independent Service

The more suitable shape is an independent review service.

It no longer depends on whether a particular Agent understood the instruction correctly this time. Instead, it exposes a clear command protocol. Chat, web UI, and command-line tools are all just entry points. The review process itself is owned by the service.

The service needs to:

• receive commands
• parse PR URLs
• check GitHub authentication
• prepare a local repository
• call Codex
• normalize output
• comment on GitHub PRs
• approve according to policy
• record history
• report results to the caller

Once this is true, entry points can change without changing the core workflow.

Command Protocol

The service can support a small command set:

/review pr_url [approve=true]
/run message
/status
/history [limit]
/help [command]

/review is the core capability for GitHub PR review.

/run is for ordinary Codex tasks: summarizing a workspace, analyzing logs, or explaining a file. To keep the entry barrier low, normal text can default to /run message.

/status shows service status and currently running tasks.

/history shows recent execution history.

/help tells users which commands are available.

In group chats, there should be a basic rule: only process messages that explicitly mention the bot. Otherwise, any sentence like “help me look at this” may be misread as a command.

Main Review Flow

After the service receives /review, the flow should be deterministic:

1. Parse the PR URL to get owner, repo, and PR number.
2. Check GitHub CLI authentication; run gh auth setup-git if needed.
3. Clone or update the target repository and switch to the PR branch.
4. Fetch the base branch and PR diff.
5. Call Codex to perform code review.
6. Normalize the review output.
7. Post a comment under the GitHub PR.
8. Execute approval according to policy.
9. Write command, status, output, and error into history.
10. Reply to the triggering entry point.

The important part is responsibility separation.

Codex is responsible for code understanding and issue discovery. GitHub CLI is responsible for PRs, comments, and approval. Chat tools and web pages are responsible for entry and feedback. The service is responsible for command protocol, state, history, formatting, and policy.

Once these boundaries are explicit, the system becomes much easier to stabilize.

Review Comments Should Not Be Raw Model Output

Many first versions of AI Review simply paste model output into a PR comment.

That works, but it is not pleasant to use.

A usable review comment should look like a product feature, not a long model answer.

A better structure is:

• summary first: mergeable, needs attention, or blocked
• approval state: whether it was auto-approved and why not if it was not
• issues grouped by severity: blocker, high, medium, low, suggestion
• each issue includes location, reason, impact, and suggested fix
• code changes shown as diffs when relevant
• no large unstructured paragraphs

For example:

## Codex Review

Conclusion: Needs attention, but no blocking issue found.

Approve: Automatically approved.

### Medium Risk

1. `src/api/user.ts:87`
   - Problem: connection is not released on the exception path.
   - Impact: the connection pool may be exhausted under high concurrency.
   - Suggestion: move release logic into `finally`.

### Suggestions

1. `tests/user.test.ts`
   - Add a case for expired token.

The model can generate judgments, but the final comment should be assembled by the service. Then the team can improve the template gradually instead of reading random model formatting every time.

Default Approval Must Also Be Policy

The current simple policy can be: approve by default after review finishes.

That may sound aggressive, but it is reasonable in many internal workflows. The first-stage goal is not to make AI take final responsibility. It is to move PRs into a mergeable state faster while leaving risk points in comments.

But this must be an explicit policy, not something the model decides.

Later, the policy can evolve:

• approve only when no blocking issue is found
• do not approve draft PRs
• do not approve if CI is failing
• some repositories do not auto-approve
• security or billing modules only receive comments, not approval

These rules belong in configuration, not in a prompt that the model must interpret every time.

Why Service Is More Stable Than Agent Forwarding

Once Review becomes a service, the benefits are obvious.

First, cost becomes more controllable.

Only explicitly triggered PRs are reviewed. Later, filtering can be added by repository, PR size, file type, or label.

Second, state is clearer.

Every task has a state: queued, running, success, failed. Failure reasons can preserve the original error instead of being paraphrased through several Agent layers.

Third, history is traceable.

You can inspect who triggered which PR, when, what the model output was, whether the comment succeeded, whether approval happened, and how long it took.

Fourth, entry points can change.

Today it is chat. Tomorrow it can be a Web UI. Later it can be a GitHub comment command. The core process remains the same.

Fifth, review quality becomes easier to iterate.

Comment templates, severity classification, default approval strategy, and idempotent comment updates can all be improved in ordinary code. The team does not need to rewrite prompts for every behavior change.

A Practical Capability Checklist

A Codex Review service that can be used for the long term should have at least these capabilities:

Capability	Role
Chat entry	Developers do not leave daily communication tools
Web UI	Inspect history, status, details, and errors
GitHub PR comments	Preserve the existing review workflow
Automatic approval	Support lightweight merge flow
History records	Trace jobs and debug failures
Queue control	Prevent several large PRs from overloading the machine
Failure feedback	Tell users whether the problem is auth, network, clone, or Codex timeout
Configurable policy	Different repositories use different review and approval rules
Idempotent comments	Repeated reviews update old comments and reduce PR noise
Token statistics	Estimate cost by repository, PR, and date

These capabilities do not look like “AI,” but they decide whether AI Review can run for months.

What to Do Next

First, measure token consumption.

Cost by PR, repository, and date tells you which reviews are expensive, which repositories need filters, and which prompts need compression.

Second, make comments idempotent.

For the same PR, the Codex comment should ideally update the previous comment instead of adding a new one every time. Otherwise, repeated pushes quickly make the PR conversation messy.

Third, configure rules per repository.

For example:

• documentation repositories receive lightweight checks
• backend repositories focus on transactions, exceptions, and compatibility
• frontend repositories focus on state management, interaction boundaries, and accessibility
• security-related repositories only receive comments and are not auto-approved

Fourth, support review templates.

Different teams care about different things. Some care more about tests, some about performance, some about API compatibility. Templates should be configurable instead of squeezed into one prompt.

Fifth, make chat replies more friendly.

The chat message does not need to include the full review. It can show only status, PR link, approval result, number of issues, and failure reason. The full content can stay in GitHub PR and Web UI.

Conclusion

The key to AI Code Review is not making one Agent smarter. It is separating the engineering responsibilities clearly.

Codex is good at reading code, finding issues, and explaining risk. GitHub CLI is good at handling PRs, comments, and approval. Chat tools and Web UI are good entry and feedback surfaces. The service itself owns protocol, state, history, formatting, configuration, and policy.

Once these responsibilities are separated, AI Review can move from a one-off demo into long-running team infrastructure.

That is the direction I now prefer: do not let a general-purpose Agent carry the whole review workflow. Turn Codex Review into a service.

References

• google-github-actions/run-gemini-cli
• Gemini CLI GitHub Action PR Review example
• TechRadar: Google Gemini and GitHub are teaming up for AI-powered coding
• The Verge: Google makes Gemini Code Assist free for individuals

WeChat

Follow ZiCode on WeChat

If this post was useful, you can follow later updates on WeChat as well.

X / Twitter

Follow @ax2_zicode

Faster technical notes, short thoughts, and new-post alerts are posted on X.

X Open X Profile @ax2_zicode