My AI filled the cart, priced it, and justified it. I glanced at it for five seconds and killed the metal in it.

My assistant does the sourcing for the metal-casting project. It finds suppliers, compares prices, reads the alloy specs, screens for lead, and stages a shopping cart with every line documented and a paragraph explaining why each item is in there. It’s good at this, genuinely better than I’d be, because it doesn’t get bored halfway through comparing tin grades.

So the cart was full and reasoned. Pewter ingot, mold silicone, a scale, and a five-pack of pure tin bars it had found cheap, flagged as a low-cost way to have extra casting metal on hand, with a whole protocol attached for verifying they weren’t lead-adulterated on arrival.

I looked at it for about five seconds and said: the tin is a hundred and seventy-nine dirhams a kilo. Are you sure? That feels like a lot of room for scrap melting.

Here’s what that five seconds did.

First, it killed the tin bars, correctly. When the assistant actually worked the question, its own research agreed and went further. The bars weren’t just unnecessary, the pewter ingot having already covered the first pours. They were the wrong metal (pure tin softens the alloy you actually want), and their below-market price was a lead-adulteration risk it had already flagged in writing, while staging them anyway. A thing it had staged with a straight face, and would have let me buy, was a mistake on three separate axes. I didn’t know any of those three reasons. I just knew the per-kilo number felt off.

Second, it changed the whole project. “A lot of room for scrap melting” wasn’t really about tin. It was me noticing that buying refined metal off a website is an embarrassing way to run a project that’s supposed to be about making things from raw material. So I made it a rule: the metal comes from scrap.

The assistant pushed back. It researched and told me scrap tin in this market saves almost nothing, that I was romanticizing it. It was right about the economics and it didn’t matter, because I wasn’t optimizing for economics. It took the constraint and re-architected the whole plan around it overnight: aluminium leads, because a bent alloy wheel is twenty to forty dirhams of raw metal and no shop drives three of them to a yard; here are twenty-nine places to go ask, none of them confirmed yet; here’s the furnace to build; here’s the one safety rule that matters, which is that water left in a wheel cavity flashes to steam and ejects the melt.

Then the honest part. I reversed the rule two days later: scrap wasn’t the real constraint, cost was. Retail metal here runs about 2.5× spot. That’s what the number had actually been telling me, and it says nothing about provenance. And none of the aluminium plan got built. The furnace sits carted and paused, unbought. No scrap in the workshop. Nothing melted. The dead tin bars are the only part of this that stuck.

Most of what I’ve read about AI is about tasks. This wasn’t a task. The assistant out-researches me, out-documents me, out-patiences me, and it staged a cart I’d have paid for. What it doesn’t have is the thing that looked at a reasonable, well-justified purchase and said no, not like this. Not because the numbers were wrong, since the numbers were fine, but because I have a sense of what the project is for, and buying the metal quietly violated it.

The uncomfortable version of this: the assistant is confidently, competently wrong, and its reasoning is airtight. The tell is a price that feels off to someone who knows what they’re actually trying to build. If I’d been a little more tired, a little more trusting, I’d have tapped checkout. The cart was right there, and it was wrong, and nothing in the cart knew it.

I’m not going to stop letting it fill the cart. It’s too good at it. But my job stopped being to approve carts. My job is the five-second veto.