Home / Benchmark / illustration
Three models got this style wrong. All seven got the colours right.
Illustration is the hardest shape target in our set: overlapping figures, irregular crops, rounded frames sitting on rounded cards. We gave seven models the same DESIGN.md, the same copy and the same output contract, and ran each one three times. Colour fidelity was 100% for every model. Shape fidelity was not.
Every model, same spec, three runs
Shape fidelity is the share of border-radius and box-shadow values in the output that match the spec. Range is the spread between the best and worst of the three runs for that model. Cells below 75% are shaded. Seven open-weight models, each on its own vendor endpoint, no aggregator, temperature 0.7, thinking disabled, three independent samples.
| Model | Shape (median) | Range | Palette | Type |
|---|---|---|---|---|
| Qwen3.8-Max | 12.7% | 43pp | 100.0% | 100.0% |
| Doubao Seed 2.1 Turbo | 24.3% | 28pp | 100.0% | 100.0% |
| GLM-5.3 | 63.6% | 42pp | 100.0% | 100.0% |
| Qwen3.8-Flash | 81.2% | 38pp | 100.0% | 100.0% |
| Doubao Seed 2.1 Pro | 100.0% | 73pp | 100.0% | 100.0% |
| DeepSeek V4.1 Flash | 100.0% | 60pp | 100.0% | 100.0% |
| Kimi K3.1 Pro | 100.0% | 0pp | 100.0% | 100.0% |
What breaks
Three places where this style leaves the model something to decide. Each one is a gap in the specification, not a reading error by the model.
- border-radius — Fractional and oversized radii on nested frames
The spec gives one radius token. Nothing states that the inner figure frame and the card beneath it must differ, so models pick one value and apply it twice. - box-shadow — Layer order and blur radius
Shadows are the one place where a model has to invent structure the spec does not name. Three of the seven models produced a single shadow where the spec implies a stack. - figure proportions — Aspect ratios on replaced content
Illustration implies crops. A spec that lists radii and colours but not aspect ratios leaves the model to decide what a figure should look like.
The finding
Illustration breaks on geometry, never on colour. Three models land below 75% shape fidelity while hitting 100% on the palette — the spec is not misread, it is under-specified where the shapes overlap.
If you build in this style, the checks that matter are on radii and shadow layers, not on the palette.
Coverage and method. This page covers one of six benchmark styles, the only six for which all seven models were run. The other 22 styles in the library were measured on a single model (Doubao Seed 2.1 Turbo, the one run across all 28 styles), so their numbers describe that model’s variance and are not cross-model comparisons. Every figure here comes from a machine check of the generated HTML against the spec’s token table — no human rating, no screenshots. Raw requests and responses are archived; how it is measured.