Field Notes
Field notes from building with teams of AI agents.
Two of three must agree: adversarial verification for agent output
An agent once reported a broken edit as verified. My adversarial verification: skeptics that try to refute, and two of three verifiers to confirm.
- Guardrails as skills: no backtest gets reported until the checks pass Guardrails as skills: my Claude Code skill won't let agents report a backtest until data validation, bias guards and multiple-testing corrections pass.
- The market was efficient after fees: an honest negative result Agent swarms tested about 1,500 hypotheses under pre-registered rules. Several edges turned out to be bugs. The market was efficient after fees.
- The prompt, not the model: what blind LLM-as-judge tests showed Blind LLM-as-judge tests showed the prompt, not the model, made an astrology app's text sound AI-written. The prompt rules and code checks around it.
- Long LLM jobs on Cloudflare Workflows: per-step retries and a budget guard Running a paid, multi-chapter LLM report on Cloudflare Workflows: one durable step per chapter, retries that skip finished work, and a budget guard.
- A score language for an AI composer Four bars of MIDI as raw JSON is about 1,600 lines, too fragile for an agent. So I designed a compact score language, a compiler and a Claude Code skill.
- Prompt caching cut a tool-use agent’s input cost by about 80% A tool-use agent resends its whole context on every turn. What prompt caching changes about that, and how I read a cut of about 80% in input cost.
- An atomic deploy pipeline for WordPress themes Atomic deploys for WordPress themes: build in GitHub Actions, lint PHP, sync to a new directory, switch with two renames, then check the routes.