My AI judge caught my AI coach cheating at the rules

Saturday night, my partner and I sat down to Dune: Imperium Uprising, a board game neither of us knew, with a rulebook neither of us wanted to read. The board was already set up when I asked my agents to build us a game copilot instead. We ended up calling it the Dune Table.

Less than ten minutes later, the Dune Table read the first photo of our cards. By the end of the night, the AI judge had corrected an AI coach ten times. And while we played, my agents shipped 21 new versions of the Dune Table.

What I built

Each of us had a private page on our phone for photos of our cards and the board. Behind the page:

• two coaches, one per player. Each coach can only see its own player's photos;
• one judge that sees everything, keeps the table moving, and writes to you privately only when someone gets a rule wrong.

A turn works like this. You take a photo of your cards. Your coach suggests a move. You play it on the real board and tap "my turn is done". Then the other coach sees the new board and starts planning for its player, before they even pick up their phone. Nobody types in the rules. The coaches read the cards from the photos and look up the rest of the rulebook.

The first version had one coach for both of us. It promised to keep each hand secret. Ten minutes later I split it in two. A secret should not depend on a promise. It should sit where the other side's model cannot read it.

The first call

In this game you use each card in one of two ways: you send it out as an agent, or you reveal it at the end of your turn. Each way does something different. My coach mixed them up.

That night the judge sent 32 private messages to my partner or to me. Ten of them corrected a coach's advice.

What the judge taught me

This is what stayed with me after the box was packed.

My coach was never unsure. Every wrong answer came in the same calm, confident voice as the right ones. If I had been alone with it, I would have played the whole night by rules that do not exist, and lost without ever knowing why.

That is the part that should worry anyone who works with AI. A model does not sound wrong when it is wrong. It sounds exactly like itself. And one agent, however good, cannot see its own blind spot, for the same reason a pilot flies with a co-pilot and nobody should approve their own pull request.

So at work I no longer use a single agent. One agent does the job. A second agent's only job is to doubt it. The agent that writes the code never approves it. The second one sees the whole picture and reads the source before it speaks. If all is fine, it says nothing.

It costs more. The judge used more tokens than both coaches together, but sent half as many messages as they did. Checking cost more than advising. It was also the only part of the night I could fully trust.

These are the judge's real instructions, translated:

If everything is fine, you write nothing.
When you are not sure of a rule, look it up before you flag it.
Flag only what you have checked, not suspicions.
You do not give strategy advice. That is the coaches' job.

The models will keep getting smarter. They will not get less sure of themselves. Trust will not come from the smartest agent at the table. It will come from the one whose job is to say: wait, check that.

Things I did not expect

• The AIs talked more than we did. 495 messages from them, 297 from the two of us. A board game turned into a group chat where we were the minority.
• I leaked my own secret. An hour in I sent a photo of my hidden intrigue card to the whole table instead of to my coach. The judge told me, privately, 25 seconds later.
• The coach owned its mistakes. After I lost, my coach listed the two mistakes that cost me the game. It sent me into a battle without counting her spice. And it forgot she could still add swords (battle strength) at the end. That is a better post mortem than most I have read at work.
• The judge would not let us call the game. At the end of the first night we were ready to announce the winner. The judge stopped us: "Don't declare the game over yet". It made us recount her coins and influence first. The result stayed the same.

My partner won 10 to 7. I lost the last battle by one sword. I blamed the judge.

The rematch

The next night we played again, and this time everyone got checked.

Night two went the other way. Halfway through, the judge told me privately to take a card out of my deck. It trusted an old note from my coach saying the card was gone. A minute later my coach corrected that note at the table: the card was still in my deck. The judge took back its own correction: "Ignore what I wrote earlier, keep the card."

When I won the last battle, the judge did not just take my word for it. It added up my points line by line before the win counted.

Nobody at that table was right all the time. Not me, not her, not the coaches, not even the judge. It did not matter, because nothing stayed wrong for long. That is my answer now when someone asks if you can trust AI. You do not trust a model. You trust the loop around it.

Oh, and I won. My partner reached 10 points first, and that ends the game after the last battle. I had 9 points and one battle left. That battle gave me three points: 12 to 10. The series is one all, and I have stopped blaming the judge.