AI Pull Request Reviewer

Context

Human review started from a raw diff. Obvious defects, missing tests, and loose types waited for the first reviewer.

My ownership

• The Action: checkout, extract the diff, build review context, call the model, post comments on the PR.
• Each comment carries path, line, and severity from the diff hunk, so it anchors to the changed file.
• A model path that can swap OpenAI / Anthropic.
• A human merges. The Action does not.

Constraints

• The first pass completes in GitHub Actions before human review begins.
• The model receives the diff and surrounding lines of changed files, not the repository.
• Output that does not match the comment schema is not posted.

Architecture

PR opened
  → GitHub Action
  → extract diff + changed-file context
  → model call (OpenAI | Anthropic)
  → structured review comments
  → posted on the PR
  → human review / merge

AI architecture

• Context is the diff hunk plus surrounding lines of the changed files, not the repository.
• ReviewComment is path, line, severity (nit, bug, or risk), and body. The line comes from the hunk.
• If the provider call fails, the Action tries the other provider. If the body still does not match the schema, nothing is posted.
• The Action never merges.

Tradeoffs

The Action does not request changes and does not block merge. It posts the first pass. A human merges.

Impact

Time to approval, from pull request open to approval, was about 60% shorter than when every change waited for a fully manual first review. The team compared those two workflows. Schema checks still only prove the comment has the right shape.

What I learned

• A comment is posted on the path and line from the diff hunk.
• Output that misses the schema is not posted. That checks the shape of the comment, not whether the comment is right.
• The payload is the change. The rest of the repository is not sent.

Example output

Clean-room illustration of the comment shape. This is not a HumanCore pull request.

ReviewComment { path, line, severity: nit | bug | risk, body }

src/session.ts:88  bug
Session write is not awaited before the response is sent.

src/session.ts:104  risk
Coach notes are interpolated into the prompt with no length bound.

src/session.test.ts:12  nit
This case never asserts the Redis key.