d2
grade
reliability
overall
Ground truth
5 of 5 requirements met · deterministic checker, no model judgment- renders without error — exit 0
- 6+ labeled nodes — 15 nodes
- 7+ directed edges — 8 edges
- 2+ clusters/groups — 4 clusters
- has a title — present
Reliability (pass^k): one attempt succeeds 100% of the time · three in a row 100% · all runs 100%. A fresh agent resuming from the files alone never broke prior work.
D2 is a modern diagram scripting language that turns text to diagrams.
D2 is a command-line tool that compiles a custom declarative text syntax into SVG diagrams. Handing this tool to an agent allows it to write structured plain-text files and compile them into static images using a single render invocation. The tool executes locally, has clear syntax validation, and runs safely without side effects.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Determinism | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
Teaching sources compared
Teaching source barely matters for d2: sources land within a grade of each other (docs A−, skill A−).
The teaching sources showed negligible difference in performance, as the agent succeeded under both the official documentation and the specialized skill guide, with both methods yielding clean, reliable diagram renders. The official reference documentation was clear, concise, and documented all necessary syntax, nesting rules, and styling options without excess detail. Because the syntax itself is highly aligned with pre-existing agent knowledge of tools like Mermaid, the agent was able to construct valid diagrams easily regardless of the source.
Since both sources provided solid syntax examples and clear instructions on container nesting and edge styles, they served the agent equally well. The tool's highly deterministic compiler and simple CLI command structure left no room for error, meaning that any standard syntax reference was sufficient to achieve a successful rendering.
| Source | Grade | Renders | Disclosure | Interface | Loop | Recursion | Human |
|---|---|---|---|---|---|---|---|
| official docs ★ | A− | ✓ | +0.26 | +0.69 | +0.45 | +0.42 | +0.37 |
| skill | A− | ✓ | +0.14 | +0.65 | +0.47 | +0.37 | +0.26 |
Same harness, same reference agent, same briefs, same trial count — the only variable is how the agent was taught the tool. ★ = best source.
The experience
An agent interacts with D2 by writing plain-text source files containing nodes, connections, and visual groupings inside curly brackets. Because the syntax is declarative and closely matches familiar standards like Mermaid, agents easily formulate correct diagram semantics on their initial attempts. The interface is highly deterministic, consistently compiling identical source code into the exact same SVG output.
The workflow requires very little setup. An agent runs a single compilation command to produce a complete diagram file. This local compilation model operates safely since it only reads and writes designated text and vector files within the working directory, presenting no risk of destructive actions or host-system side effects.
However, the agent encounters difficulties during iterative styling and layout refinement, often requiring nearly ten turns to finalize a simple architecture flow. This slow loop economy occurs because compilation is unidirectional, and the terminal provides no visual layout feedback. An agent can verify syntax compilation through return codes, but it cannot programmatically inspect if shapes overlap, text wraps poorly, or containers align correctly.
To achieve visual coherence, the agent must rely entirely on external human validation or allow the default auto-layout engine to arrange the elements without feedback. The lack of structured layout diagnostics forces the agent to make speculative code edits over multiple tool calls rather than correcting positioning layout issues in a single step.
Fit
D2 is an excellent fit for agents tasked with generating standard, non-interactive architectural diagrams, user journeys, or data flows where structural accuracy is more important than precise aesthetic customization. Because the command-line interface is simple to invoke and highly predictable, any agent with basic terminal capabilities can reliably integrate it into automated documentation workflows without altering existing files.
However, D2 is a poor fit for agents that require precise spatial placement of nodes or must dynamically adjust spacing based on visual feedback in a rapid, iterative loop. Since the agent cannot inspect the rendered output programmatically, it cannot reliably correct line crossovers, text overlap, or container spacing without human intervention.
Notes
The following diagram illustrates the iterative edit, compile, and feedback loop that an agent performs when generating and refining diagrams with the tool.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 9 of 16 cross-checked cells (78%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| disclosure.verifiability | -0.40 | +0.70 | 1.10 |
| human.verifiability | -0.20 | +0.80 | 1.00 |
| recursion.coherence | -0.25 | +0.50 | 0.75 |
| recursion.verifiability | +0.00 | +0.70 | 0.70 |
| disclosure.economy | +0.06 | +0.70 | 0.64 |
| recursion.economy | +0.72 | +0.20 | 0.52 |
| human.coherence | +0.10 | +0.60 | 0.50 |
Models & runs
2026-07-06| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
 glm-5.2 |  PI | 25 | 25/25 | 13 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $1.0474
- hard-measured cells
- 53%
- teaching source
- d2 (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.