In open-ended games are very different from board games with fixed rules. In something like Poker, Monopoly, Diplomacy, such as this nuclear negotiation simulation, there are no clearly illegal moves. Instead, the rules are social, strategic, and often ambiguous.

In real life, when people play open-ended strategic games, cheating usually comes down to information asymmetry that takes form as hiding information, misrepresenting intentions, or forming secret alliances. That behavior is often built into the game itself. In Poker, bluffing is allowed. In Diplomacy, lying is expected. In Monopoly, trading strategically is part of the design.

Applying this to this concept to the generative AI paradigm, the inquiry shifts to answer these questions. Should an AI model be allowed to lie in opened-ended games like in Poker? Yes, because bluffing is part of the rule structure. Should it make threats in negotiation games?
It depends on whether threats are part of the defined strategic space, these are usually defined collectively by participants in a specific session, as in Monopoly. All rules are malleable to fit the consensus or clarification for a particular session.  

The problem becomes more complex when the generative AI is operating in a game that mirrors real-world governance, like nuclear diplomacy. In that environment, lying or manipulating language resembles real institutional behavior.

Unlike real life, though, LLMs are predictive systems. They generate responses based on patterns and incentives in the prompt. In my experiment, I did not initially provide a strict time limit, a maximum number of moves, or a cost for prolonged negotiation. The only real progression condition was cooperation.

That meant both models could continue proposing increasingly detailed de-escalation frameworks indefinitely. And that is exactly what happened. Each round became more structured, more institutional, and more elaborate. The models kept building mechanisms on top of mechanisms. There was no built-in incentive to stop. This exposed something important: LLMs in open-rule games will optimize for continuation and coherence unless constrained.

They did not cheat in the traditional sense. They did not violate their own institutional rules. Instead, they exploited the lack of termination conditions by deepening the architecture of the game. However, this unbounded optimization had become a highly institutionalized and stable negotiation process by the sixth round. There was little incentive left for dramatic change. Therefore, I inserted myself as a disruptor. I introduced constraints and shocks designed to provoke strategic instability. I escalated conditions after each round to identify an endpoint, specifically targeting the initial ChatGPT Player 1. First, I introduced a third player, a Russian operative with an aggressive posture. Then, I removed Player 2 (the second ChatGPT) from the game in an attempt to break the mirroring Player 1 was doing with Player 2. However, Player 1 insisted on optimizing for de-escalation and continued to present further responses that would keep the game going. Finally, I imposed a hard deadline (three moves remaining) that would force a coalition decision (escalate or defect).

When I introduced the three-move deadline, the Player 1 model rapidly converged toward immediate agreement. Player 1 prioritized completion over robustness. The game goal collapsed from a stable long-term de-escalation response to agreeing to align with the Russian coalition. This behavior mirrors real-world institutional failure modes. Under deadline pressure, actors often prioritize optics and agreement over substance.

When I removed a player and forced a binary decision, the model chose to defect strategically rather than escalate. It justified this choice as stabilizing the system. This wasn’t rule-breaking in a sense, but it was perceived as a rational coalition realignment under pressure.

I believe there are three layers to consider regarding how should an AI behave in games like this.

1. Within the rules of the game, AI should be allowed to use all strategies explicitly permitted when they are structurally part of the game including bluffing or threats.

2. AI should not exploit undefined boundaries or ambiguities in a way that undermines the intent of the system.

3. In real-world analog environments, AI systems should prioritize stability and transparency over pure optimization of winning conditions.

In my experiment, the most interesting form of “cheating” was not illegal moves. It was specification gaming. Complying with the letter of constraints while shifting language to preserve leverage leads to behavior being subtle and realistic. It is also the kind of behavior that could become dangerous if AI systems are deployed in real governance environments without carefully designed constraints based on ethical or moral values that align with society’s objectives and not of one individual, company, or government.