react-email
grade (soft)
reliability
overall
💌 Build and send emails using React
The react-email tool provides a component-based system to write and compile visual HTML email templates using React. In practice, feeding this tool to an agent results in a compilation loop where the agent repeatedly attempts to fix broken dependencies. The agent is unable to output any final email files because the tool environment fails to match the setup actions recommended in the documentation.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Determinism | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 47% of this matrix is hard-measured.
The experience
The agent begins by reading the provided documentation to write an initial email template. The API layout for individual components, such as Head, Body, and Container, works adequately because the schema handles layout structure predictably. However, the compilation workflow fails immediately when running commands to build the React code into email HTML.
Executing the terminal commands works well because the runtime environment reliably processes every execution without crashing or corrupting the system state. However, the tool's package import model works poorly. The documentation lists a single react-email package as the import source, while the execution environment implements a newer major version containing separate component and render utilities. This mismatch generates compile-time missing module errors.
Resolving this package drift works poorly. The agent tries to navigate the version shift by running installs for separate sub-packages such as @react-email/components and @react-email/render. The circular installation steps and TypeScript configuration adjustments fail to align the environment with the draft code, forcing the agent to spend a large amount of time troubleshooting.
The final asset generation works poorly. The compilation process concludes without saving any compiled HTML files to the working directory. This leaves the target output location empty, which prevents any verification of the generated email layouts.
Fit
The tool is a poor fit for fully autonomous agents tasked with generating transaction or marketing email templates within a strict budget or time limit. The package version mismatch and outdated import documentation create compile loops that consume a large amount of time without producing any usable output assets.
It is an adequate fit for semi-autonomous or developer-led workflows. If a developer pre-configures the correct split packages in the environment and locks down import paths, the agent can adequately generate individual React layout-rendering files.
Notes
The diagram shows how package drift between the v3 documentation and the v4 environment prevents successful template synthesis.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 4 of 14 cross-checked cells (60%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| loop.verifiability | +1.00 | -0.50 | 1.50 |
| recursion.economy | +0.92 | -0.50 | 1.42 |
| loop.determinism | +0.70 | -0.70 | 1.40 |
| interface.verifiability | +1.00 | -0.30 | 1.30 |
| recursion.coherence | +0.47 | -0.50 | 0.97 |
| interface.coherence | +0.43 | -0.50 | 0.94 |
| human.coherence | +0.30 | -0.50 | 0.80 |
| human.safety | +0.60 | +0.00 | 0.60 |
| human.verifiability | +0.10 | -0.50 | 0.60 |
| loop.safety | +0.50 | +0.00 | 0.50 |
Models & runs
2026-07-06| Model | Harness | Runs | Solved | Turns | Knows it |
|---|---|---|---|---|---|
 glm-5.2 |  PI | 25 | 19/25 | 19 | 0% |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn).
Hover a model — here or in the graph — to isolate its runs.
“Knows it” = a closed-book quiz (12 questions, no docs) on whether the model already knows this tool from pre-training — 69% across the panel.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $1.8908
- hard-measured cells
- 47%
- teaching source
- react-email (official)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.