A score language for an AI composer

I wanted to compose music by talking to Claude in a terminal, with a MIDI sequencer in the browser playing the result. The sequencer is my fork of an open-source, MIT-licensed web app, and it saves each piece as a JSON snapshot of its song model. Four bars of MIDI as raw JSON is about 1,600 lines, too fragile for an agent. So I designed a compact .music score language, a compiler into the sequencer’s format and a Claude Code skill. I talk, Claude writes the score, the sequencer plays it.

Why raw JSON failed

The app is built with React. Under it, a core package models each song with MobX observables, Song and Track. Song has a serialize() method and a static deserialize(), and a saved piece is the serialized form. Each track keeps its events in an _events object. In the compiled output, every event appears twice: once in array and once as a copy in lookupMap.

{
  "id": …,
  "channel": …,
  "_events": {
    "array": …,
    "descending": …,
    "lookupMap": …,
    "lastEventId": …
  },
  "endOfTrack": …
}

My first setup told Claude to edit that JSON directly. The first demo piece was a hand-written JSON file of 82 lines. It stored events as a plain array instead of the _events.array shape the serializer expects, so the notes never loaded.

The fix built the demo inside the browser, with the app’s own Song, conductorTrack, emptyTrack and track.addEvent, then serialized and saved it. The commit says this guaranteed that “the on-disk JSON matches what Song.deserialize() expects”. But the music was now code written against the app’s API. The compiler commit that followed is blunt: hand-editing the JSON “is unworkable”. The compiler replaced the in-browser builder.

The score language

A domain-specific language, or DSL, is a small language built for one kind of problem. This one describes music as plain text, line by line. A .music file opens with optional header lines such as name, tempo, key and timesig. They are only read before the first [Track] block. The defaults are 120 BPM and 4/4, and the name falls back to the file name. The key line is informational. Here is part of the first score:

tempo 120
key Am
timesig 4/4

[Lead] prog=81
| A4 C5 E5 D5 | C5 ~ ~ ~ |
| E5 D5 C5 .  | A4 ~ ~ ~ |
| C5 E5 G5 A5 | G5 ~ ~ ~ |
| E5 D5 C5 .  | A4 ~ ~ ~ |

[Drums]
kick:  X . . . x . . .
snare: . . X . . . x .
hat:   x x x x x x x x

Each bar sits between pipe characters. The compiler splits a bar into equal steps by counting its tokens: 4 tokens make quarter notes, 8 make eighths and 16 make sixteenths. So the grid can change from one bar to the next. A token is a pitch such as A1, C#3 or Eb2, a rest (.), a tie that holds the previous note (~), or a chord joined with plus signs, such as A2+C3+E3. Pitch names map to MIDI note numbers, with middle C as C4 = 60.

Track headers take prog= for the General MIDI program and ch= for the channel. A track whose name contains “drum” goes to MIDI channel 10, which General MIDI reserves for percussion. Drum lanes are name: steps lines on an eight-step grid, one bar of eighth notes. Twelve lanes map to General MIDI drum notes: kick is 36, snare 38, closed hat 42.

The compiler is strict in one place and loose in another. A malformed pitch stops the build with a Bad pitch error. An unknown drum lane name is skipped without an error, so a typo there goes unreported.

The compiler

The compiler is tools/compile.mjs, a single Node ES module that imports only fs and path. It was added with 358 lines and now ends at line 452. It runs as node tools/compile.mjs songs/<slug>.music.

Writing the exact shape

The commit that added it says it compiles the score to the sequencer’s exact JSON. Song and track objects carry the fields of the core Song and Track schemas. Track 0 is a conductor track: a name, a time signature and a tempo in microseconds per beat, Math.round(60000000 / score.tempo). Every instrument and drum track then gets a fixed list of controller, RPN and pitch-bend events at tick 0. The source comment says this mirrors the app’s emptyTrack(channel). Program changes go only on non-drum tracks. The timebase is fixed at 480 ticks per quarter note. The output file name uses the same kebab-case slug rule as the dev server.

The check is the round trip. The compiler commit records it: “Verified end to end: the generated JSON round-trips through Song.deserialize() and songToMidi().” Those are the app’s own deserializer and MIDI export. The first is what Load from Disk calls. The second writes the MIDI file.

Three fixes

Drum lanes were first compiled in a single pass. A one-bar pattern only sounded in bar 1 while the melody ran the full length. The fix loops each drum pattern to cover the longest melodic track, “like a DAW”. A 16-step lane loops as a two-bar unit, and hits past the end of the song are dropped.

The comment stripper first treated every # as the start of a comment. That removed the sharp from pitches like C#3, and the line became | C. Now # starts a comment only at the start of a line or after whitespace: /(^|\s)#.*$/. The fix shipped in the same commit as a piece in C# minor.

The output also sounded flat. The commit says why: “every note had fixed velocity and landed dead on the grid”. Melodic notes had velocity 95, and drum hits had 120 or 95. The fix added global swing and humanize, per-note velocity (A2:110), per-track vel= for velocity and gate= for note length, and arp(...) tokens that spread a list of notes one per step across a bar. Drum hits became three types: X accent, x normal, o ghost. An accent is the track velocity plus 25, capped at 127. A ghost is the track velocity minus 45, with a floor of 20.

Swing delays the off-beat steps by a fraction of a step. Humanize shifts timing and velocity, scaled by the humanize value. It does not use Math.random or Date.now. It uses a deterministic function of an integer seed, “so humanize is reproducible”. The skill tells Claude to set swing around 0.15–0.2 and humanize around 0.3 for hip-hop and trip-hop.

The skill and the loop

The last part is a Claude Code skill called compose. It lives outside the repository, in two files: SKILL.md and reference.md. It triggers on any request to create, add, change or generate music, or to work on “the piece”, “the track” or “the song”. Its workflow has four steps: read CURRENT_PIECE.md, edit songs/<slug>.music, run the compiler, and tell me to reload. A new piece is a new .music file. The reference file covers note numbers, scales, chords, progressions, General MIDI programs and the drum map.

The dev server rewrites CURRENT_PIECE.md whenever a piece is loaded or saved. It lists the tempo, time signature and timebase, and for each track the channel, program, note count and pitch range. Its header says Claude reads it to know which song is active, and that it is not to be edited by hand. The skill and the repo’s CLAUDE.md both forbid guessing the active piece: Claude lists the scores and asks. If a musical feature is missing, both tell Claude to extend the compiler rather than hand-write JSON.

A Vite plugin adds /api/pieces to the dev server, to list, load, save and delete pieces on disk. Its comment gives the purpose: Claude Code edits pieces on disk from the terminal, while the browser app saves and loads them through these endpoints. The app does not hot-reload pieces. After a compile, I pick the piece in File → Load from Disk and press Play. From there, the workflow in CLAUDE.md ends with .mid files exported to open in an external DAW.

This is one of the agent systems on my home page. Its whole git history, from the fork import to the last compiler change, falls between 12:11 and 13:23 on 30 June 2026. Every commit carries a Claude co-author trailer.

What I’d keep

• A text format for the agent, and a compiler for the machine format. Claude edits the score, never the generated JSON.
• Checks that use the app’s own code. The round trip goes through the deserializer and MIDI export the app itself uses.
• Copying the app’s defaults instead of guessing them: the tick-0 events of emptyTrack, the slug rule of the dev server.
• One rule for gaps. When the language lacks a feature, extend the compiler. Don’t hand-write JSON.
• Determinism. Humanize runs off a seeded function, so it is reproducible.