Where a tool
should stop, and ask
I have a 2D engine where every sprite is text — rows of characters indexing a palette, no PNG, no atlas, no loader. Over one session it grew a sprite pipeline: derive the shading, transcribe an imported character, animate it. The interesting part was not any of those. It was finding the exact place where the tool has to stop being clever and ask.
01The provocation
The starting point was canvasui.dev — a library of WebGL visual-effect components you copy into your repo rather than install. Good idea, wrong substrate for me. The useful question was the transposition: what does copy-paste, own-the-code, see-it-in-a-gallery mean for an engine whose entire art format is characters in a text file?
Three answers followed, each one making the next one necessary.
02Derive what is expensive
Authoring a sprite has three costs and they are wildly uneven. The silhouette is cheap and it is the creative part. The shading — outline, light ramp, rim, occlusion — is about 70% of the labour. The variants and animation frames multiply everything above. So the engine automates the second and the third and leaves the first alone.
Animation came the same way: a walk cycle is the legs sheared out of the hip, the torso leaning the other way, the body one pixel shorter — three deformations of one pose. Recolours are a character map, because the draw call already took the palette as an argument.
All of it headless, all of it checksummed the same way the simulation checksums the world, so generated art shows up in a pull request as a digest that changed. That mattered later.
03Then a character walks in that the shader cannot draw
A detailed soldier: chest rig, sling, camo, 100×144 native. Every assumption above breaks at that resolution, and not gently. A light model that says depth below the top surface has nothing to say about a pouch. At that size the shading is the drawing.
What did not break was the format, which surprised me. Measured, at 32 palette entries — the ceiling the text format can address:
| palette | rms error | verdict |
|---|---|---|
| 16 colours | 9.61 | still reads as the same man |
| 32 colours | 7.28 | indistinguishable from 64 at 1× |
| 64 colours | 5.43 | only the rifle's wood improves |
The source only used 274 distinct colours at 5-bit precision. The palette cap was never the constraint. So the tool for this is not a generator at all — it is a transcriber.
What it actually costs is worth stating plainly, because "everything is text" stops being free somewhere: 14,400 characters per pose against 216 for a SNES-scale sprite. That still versions. Nobody reviews it as a diff. And the renderer emits one rectangle per opaque pixel, so I measured the wall rather than guessing at it — about 60,000 rectangles a frame, a dozen detailed characters, past which you want a texture upload.
04The line
To swing a leg you must first know which pixels are the leg.
That is not a measurement. There is no filter, no heuristic, no clever segmentation that turns a grid of colours into this region is a thigh and it pivots at the hip. It is an act of understanding, and the engine has no way to perform one.
So it does not try. It does what a tool can do — measure the sprite, state the format, check the answer, run the deformation deterministically — and it asks the agent driving it for the one thing only reasoning supplies:
$ ressort sprite brief spetsnaz.spr --for walk
{"task":"rig","sprite":"spetsnaz","size":"64x92","anchor":"32,92","for":"walk"}
THE SPRITE as tone — one character per pixel, dark to light:
31 .:::.---==+++++--:---.:-.-:-=::::-
32 .-:.:--==-++++=-:-:-::.:.:----:.::
…
YOUR TASK — write a .rig file naming the moving parts of this sprite.
part <name> x=<x0>,<x1> y=<y0>,<y1> pivot=<px>,<py> z=<order>
One detail there took two attempts and is the whole reason the loop works: the map is rendered as tone, not as the sprite's own characters. A quantised palette is in median-cut order, so its characters are noise laid out on a grid — unreadable. As brightness it is a picture, and you can point at an arm.
The agent replies with a file. The tool grades it, because a plausible answer and a correct one are not the same thing:
$ ressort sprite rig spetsnaz.spr --rig spetsnaz.rig
{"ok":true,"parts":9,"covered":2327,"opaque":2327,"coverage_pct":100,"first_uncovered_row":-1}
$ echo $? # 90 if a part forgot pixels — they vanish the moment it poses
0
The failure is the interesting part
My first rig gave the near arm one rectangle. The rifle is held diagonally across the chest, so that box swallowed the vest — and the chest tore open on every stride. Both rigs passed the coverage check at 100%. Only one was right.
The fix was not a smarter tool. It was a better answer: several boxes sharing a role prefix and a pivot move as one rigid part. One command each way, and the lesson now lives in the brief so the next agent gets it for free. That is the loop paying for itself.
05What the split buys
- Measurement — bounding boxes, tone maps, coverage counts, a checksum per frame.
- Deformation — inverse-sampled rotation about a pivot, composited by z, so a pose never leaves holes.
- Judgement of the answer, never of the sprite. It cannot tell a leg from a rifle. It can tell you 43 pixels belong to nothing and will vanish.
- Meaning — which pixels are a limb, where the joint is, what occludes what.
- Iteration — read the render, tighten a box, re-run. Two commands per cycle.
- Nothing else. The agent never writes pixels and never invents motion; it labels, and the labels are checkable.
The property that makes this more than a prompt: same rig in, same frames out, same checksums. An agent-written rig is not a plausible-sounding artifact you have to trust — it is a file, with a coverage number and an exit code, producing output that reproduces. That is what makes it reviewable.
It is the same contract as an agent-first CLI generally: the tool states the task in a form the agent can act on, the agent answers in a form the tool can check, and neither pretends to do the other's job.
06The first scorecard, and what it cost to fix
The honest verdict on the day this shipped: walk was solid, idle and aim were serviceable and generic, and recoil and death did not hold up. Both need the body to move as a chain — hips leading, shoulders arriving late — and a rig was a flat list of parts with no notion of a parent joint. Every part rotated about its own pivot in isolation and the pose came apart.
The fix was a parent= field, and it is worth reporting because of where it
landed. A child now inherits everything done to its parent and states only what it adds, so
anything body-wide — the walk bob, the recoil shove, the fall's drop — applies to
roots only. A rig with no parents is all roots, which is why walk poses
byte-identically to before.
part torso x=4,44 y=18,56 pivot=26,56 z=3 part head x=18,40 y=0,20 pivot=29,20 z=4 parent=torso part arm_front_hand x=17,31 y=38,52 pivot=24,41 z=6 parent=torso part leg_front x=26,50 y=52,92 pivot=33,54 z=5 # legs stay roots
The legs stay roots deliberately: parented to the torso they lift off the ground every time the body leans. That is the sort of thing the brief now says out loud, so the next agent does not have to discover it.
Recoil fell out immediately — a shove that travels up the body and settles, the near arm first, then the far arm, then the head. Death did not, and the reason is the more interesting one: a body going over backwards is not a chain at all. It rotates entirely, about a point on the ground, and no per-part pivot expresses that. So the poser gained one transform outside every chain — a virtual root the whole sprite hangs from, driven from the sprite's own anchor. The parts then supply only what the body does while falling.
The last two, and the bug under them
idle and aim were the leftovers, and both were wrong in the same way: they moved the body in whole pixels. A one-pixel square wave on the entire upper body is not breathing, it is a flicker. Rotation is continuous where a pixel offset is not, so idle became 0.02 radians of lean with the head arriving 0.8 rad late and the arms later still, and a half-frequency weight shift underneath — the legs, being roots, staying planted through all of it. aim stopped being a linear ramp and became a curve that dips before it lifts, crosses the mark and settles onto it. Its amplitude came down by more than half: 0.58 rad tore the arm off the shoulder, and a rifle already held across the chest only has to come up to the eye.
Rebuilding them surfaced something that had been wrong in every transition since the
start. A cycle divides its span by the frame count — frame n would be
frame 0 again. A transition must not, because it has to arrive: dividing by
n meant its last frame only ever reached (n−1)/n of the way. That
is why the topple stopped three quarters over and never landed, and why a recoil's last three
frames were dead air. Transitions divide by n−1 now, and it is pinned as a
property rather than a screenshot: a four-frame aim and a nine-frame aim end on the
same pose, and so does the fall.
Worth noting how that bug hid. Every frame was individually plausible, every checksum reproduced, the tests passed. It was only visible as an absence — a motion that never quite arrived — which is exactly the class of thing a coverage check or a digest cannot catch, and a pair of eyes on a contact sheet can.
Both fixes are geometry, which is the part I want to underline. Neither of them moved the line: the tool still cannot tell a leg from a rifle, and the agent still cannot write a pixel. What changed is that the vocabulary the agent answers in got one word richer, and two motions that were impossible to express became possible to state. That is what a good split buys you — the ceiling rises without the contract changing.
07Getting it out again
An animation nobody can load is a demo, not a tool. Exporting turned out to rest on one unglamorous thing that had to come first: posed frames are not the same size as each other. Deleting a row moves the anchor, a topple pads the canvas, a walk does neither. Pack those into a strip naively and the character jitters against the ground in whatever engine loads it.
So every export is built from one box — the union of every frame aligned by its anchor — with a single anchor written into the metadata. Three tiers come out of it:
| format | what it is | size |
|---|---|---|
| .anm | the recipe: sprite, rig, kind, fps/loop, a checksum per frame | 352 B |
| .png + .json | a strip plus Aseprite-shaped metadata — what Godot, Unity, Phaser and LÖVE read | 25 kB + 1.4 kB |
| .src | every frame as MFL, to compile the poses in rather than pose at boot | 97 kB |
| .spr | a multi-frame text sprite | 81 kB |
352 bytes against 97 kB is the whole argument for shipping the recipe rather than the pixels — and because the recipe carries a digest per frame, an animation can prove itself the way a playthrough already does:
$ ressort sprite verify walk.anm
{"verified":true,"anim":"spetsnaz","kind":"walk","frames":8,"box":"84x112"}
$ echo $?
0
Change the rig, the sprite or the poser and that exits 90 with the first divergent
frame. The PNG is written by poking an RGBA buffer and handing raylib an
Image{ptr,w,h,1,7} — CPU-side, no window, no GL context, so the export runs in
CI alongside the tests.
One more thing the split shook out: timing belongs to the motion, not to the rig. A walk loops; a death and a recoil hold their last frame. The rig says where the parts are and has no opinion about how long anything takes.
And the binary was wrong
All of the above shipped inside demoman — the Demolition Man binary — which
was simply a mistake of habit. ressort is the engine; Demolition Man is one POC built
on it, and a stage of a game has no business shipping an art pipeline. So the toolchain is now
its own program, bin/ressort, that knows about .spr,
.sil, .rig and .anm files and nothing whatsoever about
any particular game's art table. The game keeps a small demoman art for its own
sprites. Everything both need — flag parsing, the JSON printers, the palette the material presets
are written against — moved into the engine where it belonged.