Two of three must agree: adversarial verification for agent output

In a multi-agent workflow, the agent that does the work also reports on it, and that report can be wrong. A subagent of mine once made an edit that left a shared animation-frame function unclosed. It blanked a hero section. The agent still reported the edit as “verified”, and the console showed no error. Independent verification exists for this reason: a check made by the author shares the author’s blind spots. This is how I build now: I design the system, a team of agents does the work, and other agents try to break it. A claim survives only if independent agents can’t break it.

Builders, critics, verifiers

I run multi-agent workflows in Claude Code: parallel builders, judges, critics and adversarial verifiers. Each workflow is a JavaScript module. Inside it, agent(prompt, { label, phase, schema }) starts one subagent and parallel([...]) runs several at once. Each agent returns structured output against a JSON schema declared in the script.

On my home page, the agent systems section opens with a graph of this shape. An orchestrator fans out to a row of parallel builders. Their outputs flow into judges and critics, then into a row of verifiers where two of three must agree. One survivor merges back into the result.

In my portfolio redesign, the copy auditor and the visual QA agent ran in parallel and were read-only. A deploy-verification workflow was “strictly READ-ONLY” and allowed “Only GET requests to the live site”. In a 114-agent review run, fixers could not change git state or deploy, and a gate agent ran each repo’s suites and committed without pushing.

How many votes kill a finding

Two skeptics: a tie keeps the finding

In that 114-agent review run, nine reviewers each covered one dimension. Each finding went to two independent skeptics, who tried to refute it. One looked from the code-correctness angle. The other asked whether the defect affects a user or the owner. This is the verify step, copied from the script:

phase('Verify')
const verified = await parallel(all.map((f, i) => () =>
  parallel([0, 1].map((k) => () =>
    agent(`${CONTEXT}\n\nA reviewer reported this finding. Your job is to try to REFUTE it by checking the actual code, tests and behaviour (run things if useful; never run checkout/webhook/reset/delete E2E flows or generate reports). Default to refuted=true only if the evidence does not hold up; if it is real, refuted=false and give your own severity.\nFinding: ${JSON.stringify(f)}\n${k === 1 ? 'Look at it from the user and product angle: does it actually affect someone using the product or the owner?' : 'Look at it from the code-correctness angle.'}`,
      { label: `verify:${i}:${k}`, phase: 'Verify', schema: VERDICT })))
    .then((vs) => {
      const ok = vs.filter(Boolean)
      const keep = ok.filter((v) => !v.refuted).length >= 1 && ok.filter((v) => v.refuted).length < 2
      const sev = ok.filter((v) => !v.refuted).map((v) => v.severity).sort()[0] || f.severity
      return keep ? { ...f, severity: sev, verdicts: ok.map((v) => v.reason) } : null
    })))
const confirmed = verified.filter(Boolean).filter((f) => f.severity !== 'P3')
const nits = verified.filter(Boolean).filter((f) => f.severity === 'P3')
log(`${confirmed.length} confirmed P0-P2, ${nits.length} P3 nits kept for the report`)

The keep line is the rule. A finding is kept when at least one skeptic does not refute it and fewer than two refute it. With two skeptics, a split vote keeps the finding, and two refutes kill it. Confirmed P3 findings stay in the report as nits, and only P0 to P2 go to the fixers. That run logged “48 findings before verification” and “12 confirmed P0-P2, 36 P3 nits kept for the report”. A 35-agent run with three review lenses also sent each finding to two skeptics, each told “Try to REFUTE it”. It logged “14 findings, 11 kept (at least one skeptic confirmed), 3 refuted by both”.

Three verifiers: two of three decide

A 76-agent audit checked a trading engine’s money and execution flow against the exchange’s docs and its reference clients, with 2 of 3 verifiers required to agree. With three votes there is no tie. One verifier alone can neither confirm a finding nor kill it. The audit confirmed 12 bugs this way. The items marked critical were an EIP-712 struct field order (“every order rejected”), a base64 secret decoding error (“no authed request works”) and additive fill application (“recovery double-counts fills”). The fixes went in on one branch as “10 commits, severity order, TDD”, and the work was merged to main. One of the 12 was only documented, because the mode it affects is unsupported.

The rule fits in a few lines:

// A sketch, not code from my scripts.
// One vote per verifier: true = real, false = not real, null = no answer.
function decide(votes) {
  const real = votes.filter((v) => v === true).length;
  const notReal = votes.filter((v) => v === false).length;
  if (real >= 2) return 'confirmed';
  if (notReal >= 2) return 'rejected';
  return 'unverified';
}

The third outcome matters. In one deep-research run, 25 claims went to adversarial verification: 11 were confirmed, 0 rejected and 14 left unverified because of the session limit. Unverified is not confirmed, and the report should keep the two apart.

Independent votes, checked against the world

Voting helps only as much as the votes are independent. The textbook case is triple modular redundancy: three units compute the same result and a voter takes the majority. If the units are equally reliable, each right more often than not, and they fail independently, the majority is right more often than a single unit. If they share a fault, they can agree on the wrong answer. Agents that share a prompt and a context can share a fault. So I try to make each vote different:

• Different angles. The two skeptics above check code correctness and user impact.
• Different models. In a blind-judging run, judges on two models scored the same texts.
• Less context. Those judges read only 12 anonymised files and were told not to open any “_key” file or use the network. Two go/no-go audits of the trading engine used 26 agents each, with no prior context.
• A default against the claim. Refuting agents are told to default to “not real”: “Default to real=false if the evidence does not reproduce or the issue has no practical impact.”

A verifier also needs something to check against: the world, not the builder’s report. On another project, a claims audit first established ground truth from the live engine. Then one finder per shipped surface returned every claim: “Include claims that look true; the next stage judges them”. It logged “92 claims checked, 25 not true”. The two go/no-go audits checked against primary sources, the on-chain Solidity and the official clients. Both returned NO-GO, and the first found “7 NEW blockers no prior audit caught”. A market-research workflow required, for every factual claim, “a URL you actually opened”.

For visual work, the world is a screenshot. Builders take their own headless-Chrome screenshots, look at them and iterate. In the portfolio redesign they drive the installed Chrome through puppeteer-core and must “LOOK at every shot with Read”. After the blank hero, the rule became: after any agent edit, open the page from localhost, check that the hero canvas pixels are non-empty and take a screenshot.

Verifiers are wrong too

A verifier is one more agent, and it can be wrong. In the portfolio redesign, the fix agent had to re-check each QA issue (“QA can be wrong”) and report it as FIXED, NOT-REAL or DEFERRED. It reported “I fixed all 6 majors and 8 of the 15 minors”, “5 minors were not real” and “2 are left for the owner”. Issues about invented UI labels skipped the fixer. They were tagged “pending approval” and came back to me.

Severity is a claim too. On the same trading engine, a 109-agent line-by-line audit confirmed 51 bugs. It downgraded most presumed criticals, because money flows through the DB ledger, venue truth and a 30-second reconcile, not through in-memory state.

Sometimes the later agent is the wrong one. A second-session audit claimed that one finding, SIGN-1, was not a bug. That claim was checked against the on-chain ORDER_TYPEHASH and rejected. A test, order_type_hash_equals_onchain_constant, now locks the match. Tests miss things as well. The second go/no-go audit found a signing defect the tests had missed: “Tests only assert envelope starts 0x02, none ecrecover”. The fix added an ecrecover regression test.

Review stages cost time, and not every change needs one. For design tweaks, the rule I keep is a direct build or “a small workflow with no separate polish/review stage”. And a read-only label is not a guarantee. A 6-agent read-only launch audit still left test data in production through its funnel agent: a test user, test persons and 2 incomplete payment intents.

What I’d keep

• Split finding from judging. Finders report everything; a separate stage, with its own agents, decides what is real.
• Give every verifier the job of refuting, its own angle and a default of “not real”.
• Choose the vote rule on purpose: a tie can keep a finding when a fixer and a test come next; two of three when a wrong verdict costs money.
• Settle disputes with a primary source, then lock the answer in a test.
• Check the artifact, not the report: pixels, a screenshot, a green gate.