How do you design a schema a language model can emit reliably?

I maintain svelte-a2ui, the Svelte renderer for A2UI — the protocol where an agent describes an interface as data against a component catalog the host application owns. No generated code, no iframes, no HTML from the model. The renderer was deliberately the boring part: track the spec, stay out of the way.

The interesting design problem lives one layer up. A catalog is a vocabulary — the set of component names, props and enums an agent is allowed to speak. And the users of that vocabulary are not developers reading documentation at their leisure. They are language models emitting JSONL, token by token, mid-stream, with no backspace key.

That changes what "good API design" means. So when I started auri — ready-made catalogs for A2UI, beginning with an ops catalog of stats, charts, tables and approval flows — I set one rule before anything else.

No component gets implemented until language models can emit its wire format cleanly, cold. If a model fumbles a prop shape, the contract is wrong — never fix it with prompt engineering.

The reasoning is about where a fix lands. An awkward prop is a tax paid at inference time, by every agent, in every deployment, forever. A prompt workaround fixes one deployment. A contract fix fixes all of them. So the contract absorbs the blame, always.

The gate

The method is almost embarrassingly cheap:

1. Draft the contract — a JSON Schema of components, props and enums — and a prompt-pack, the system-prompt snippet that teaches the vocabulary, with a few known-good example streams.
2. Paste the pack into a fresh session of at least two model families. No context, no warmup.
3. Ask for the output for a realistic scenario: "show the on-call engineer an incident view".
4. Validate what comes back with the same schema validator your CI uses.
5. Every failure is a design review comment. Iterate the contract, not the prompt.

Later the gate stopped being a ritual and became a script — a harness that sends the pack cold to a model matrix and scores every emission against the contract. It runs in CI now, and it uses the same validator as the fixture tests, so the contract, the documentation and the evaluations cannot physically drift apart.

I went in with principles I was fairly confident about. Flat props, because a model streaming JSONL should never have to juggle deep balanced structures. Small closed enums with forgiving defaults. Semantic and never visual — intent: good | bad | warning | info | neutral, never colours. Raw values on the wire, 12400 rather than "$12.4K", with the component formatting via Intl in the reader's locale. And accessibility as required schema props, because agent-composed pages cannot be audited before they exist.

The principles held up. What I did not expect was what the models did with them.

What the models taught me

Models discover affordances you did not design. Promote the good ones. My Stat component had unit as free text, and I had ms, % and req/s in mind. First cold run, a model emitted unit: "USD" for a revenue tile — a currency code, expecting locale-aware formatting on the other end. That is better than what I designed. The contract now explicitly blesses ISO 4217 codes there, and the component formats them as currency. When a model invents a sensible usage, that is not an error to correct. It is a feature request from your most important user.

A model can follow your rules more consistently than your contract does. My rules said raw values on the wire — ISO timestamps, never pre-formatted strings. Then a model emitted ISO timestamps as chart x-axis labels, which my contract rendered as given: raw ISO strings on screen. The model was right and the contract had a gap, which is why xFormat: "datetime" exists now.

It happened again with Progress. My principles said everything displayable accepts data bindings, and a model bound the progress denominator to data — "max": {"path": "/totalPods"} — for a rollout where the pod count could change. My schema said max must be a plain number. The schema was violating my own design language, and the model caught it. Twice, across two model families, the "error" was the contract being less principled than its users.

Some failure modes are format-level, and a sentence fixes them where a schema cannot. One model family kept producing lines like …"value":[4.2,3.1]}]} — closing every visible structure but dropping the envelope's final brace. Each A2UI line is {"version":…,"updateComponents":{…}}, so there are two closers minimum. No schema change can fix brace-balancing.

What fixed it was naming the trap in the prompt-pack. One sentence — every line ends with two closing braces minimum, and a dropped final } is the most common emission mistake — took that family from failing half its hard scenarios to a clean sweep. Traps like this transfer across model families. Write them down where every agent will read them.

The loop runs backward too: implementation findings feed the contract. My approval component had onApprove and onReject action props. The first browser test failed mysteriously, because the renderer correctly strips every on*-prefixed prop as a defence against smuggled event handlers — the reasoning is in agent-generated UI is untrusted input. Those props could never have reached any component, in any renderer with the same defence. The contract renamed them to approveAction and rejectAction, and the design language gained a standing rule: never name action props like DOM event handlers. The security model of your runtime is part of your vocabulary's design constraints, whether you noticed or not.

If you can make the mistake, a model will. Writing my own demo, I sent a data update without a path — which, per the protocol, replaces the entire data model. Half the dashboard collapsed to loading skeletons mid-demo. I am the person who wrote the contract and I still hit it. That is not an anecdote about being careless; it is evidence about the shape of the API. The prompt-pack warns about it now, in the rules section, where every future agent will see it.

The prompt-pack is half the product

The strangest realisation was that the system-prompt snippet is not documentation about the product. It is the product, as much as the schema is. An agent cannot use a vocabulary it has not been taught, and the quality of the teaching text decides emission reliability directly.

So the pack ships as a versioned artifact next to the contract, and the documentation site renders its sections verbatim as the component reference, with each component's example stream doubling as its live demo. One artifact, two audiences — and if the documentation does not work as few-shot material, the documentation is wrong.

If you are designing any schema for models

None of this is specific to interfaces. It applies to tool definitions, structured-output schemas and DSLs — any JSON you want a model to emit reliably.

• Put the schema in front of a model before you build what is behind it. It is the cheapest design review available, and almost nobody does it.
• Treat consistent model errors as design feedback, not model failure. Especially when the model is following your stated principles better than your schema does.
• Never patch with a prompt what you can fix in the contract. Prompt fixes are local; contract fixes are universal.
• Ship the teaching text as a versioned artifact next to the schema, and evaluate against it.
• One validator for tests, evaluations and documentation. Drift becomes impossible rather than unlikely.
• Write down the traps. Format-level failure modes repeat across model families, and a sentence in the right place is worth more than a retry loop.

auri is live at chaliceforauri.github.io/auri — the front page is an agent building an incident console, with the wire shown beside it. The contracts, prompt-packs, emission harness and the full gate log — every finding above, with dates and diffs — are in the repository.