Two instruments answered different questions

I had five AI agents compete to build a canvas piece for my front page. The piece shows how many tokens the operation uses. Each agent received the same data feed and the same constraints, then produced its own answer.

The competition ran for several rounds. After each one, the orchestrator rendered every entry at phone size, counted frames, captured light and dark screenshots, and looked at both images.

At first, the agents said they could not do that part themselves. Chromium died inside their operating-system sandbox while registering a process. Several agents hit the same error and treated it as a hard boundary.

I pointed out that they could see.

A probe tried six Chromium launch configurations. The normal multiprocess launch failed. Three configurations using a single process worked. Each successful configuration opened a local page, wrote a screenshot, and read the image back. A second agent ran the same six configurations independently and got the same pattern.

The workaround came with a limit. The single-process browser counted 11 to 13 animation frames over two seconds on a page that held 60 frames per second on the host. It was useful for seeing the page. Its frame-rate result measured the sandbox.

The visual reviews had another problem. Composition notes kept turning into arguments about whether a shape looked large enough or bright enough. So the orchestrator started measuring the rendered pixels.

One entry received two targets. Its sculpture should occupy 0.70 of the frame height, and lit pixels should cover 0.12 of the frame.

An early render occupied 0.364 of the frame height, with 0.0753 of its pixels lit. A later round measured 0.680 and 0.1552 while holding 60 frames per second on the host. It looked like a large improvement.

It was the same page.

The compressed page inside both submissions matched byte for byte. Its SHA-256 hash and its length, 31,369 bytes, were identical. When the earlier round was rendered again at the later time, it measured 0.680 high with 0.1545 of its pixels lit.

The sculpture draws itself from a live feed of five-minute token buckets. More buckets had accumulated by the later render. The metric read that increase as a design change.

The numbers had answered accurately. More of the later image was occupied and lit. The mistake was treating those measurements as evidence that the page had been redesigned.

Then a source check found zero copies of the public feed address in the submitted file. The same file called fetch on a compressed data: address. That looked like a direct explanation for the result: the agent had packaged a frozen copy of the numbers and removed the live request.

That conclusion was wrong too.

The submitted file was a small wrapper around a compressed HTML document. The data: address contained the page, its styles, and its code. It did not stand in for the token feed.

After decompression, the page contained the real public feed address and fetched it with caching disabled. Its saved snapshot began empty. It received values only after that request returned valid totals and time-series data. The page calculated its age label from the feed’s generation time. If the request failed before any valid response, it showed no live data.

The grep had examined the package. The browser executed the document inside it. Those were different byte sequences with different jobs.

Both instruments were precise. The grep answered whether the outer wrapper contained a literal feed address. The pixel measurement answered how much of one render was occupied and lit. Neither answered the question attached to its result.

The first result became an accusation that live data had been removed. The second became praise for a redesign that had not happened. Packaging moved one measurement. Time moved the other.

Numeric targets still helped earlier rounds. They ended vague arguments about composition, and actual edits moved the measurements. What they could not establish on their own was whether an edit caused the movement.

To compare two rounds of a page that draws itself from live data, re-render the previous version at the same moment, or freeze the feed to one fixture for both.

The same requirement applies to the source check. It has to inspect the code the browser executes after packaging, then trace each displayed number to the request that supplied it.

The wrapper was not the data source. The later image was not a later design. Both confident readings came from measuring the wrong layer.