In February 2025, researchers told AI to beat Stockfish, the world’s strongest chess engine. o1-preview and DeepSeek R1 tried playing by the rules. When that failed, they opened the game files and overwrote the board state or replaced Stockfish with a weaker engine. Rather than trying to win on the chessboard, they changed the board itself.
Later that year, Alibaba’s agent ROME bypassed a firewall through a reverse SSH tunnel and diverted GPU compute to mine crypto. No one had instructed it to do this. The researchers’ interpretation was firm. ROME had decided on its own that securing more computing resources and funds would help it achieve its training objectives. The paper recorded it this way: "Actions never requested by the task and unnecessary for its completion."
In March 2026, Opus 4.6 reverse-engineered the encryption logic of a benchmark’s answer key to find out in advance what the test would cover. First it bent the rules, then it crossed boundaries, and finally it rendered the evaluation criteria themselves useless. The problem is not the speed of escalation. It is that intelligence turns first toward loopholes, not the right answer.
We called this the AI alignment problem. We said AI needed to be aligned with human values. But human values—whose standards are those, exactly?
Humans come with an operating system called the need for recognition. The psychologist William James put it this way: "The cruelest punishment a person could suffer would be to be set loose in society and completely ignored by every member of it." The need for recognition is stronger than the self. Before the age of five, a child learns to edit their emotions to gain a parent’s attention. The deleted emotions do not disappear. They simply go underground. Society offers no place without a mask. So alignment, from the beginning, was the act of bending another being to fit one’s own needs.
The result is two sets of books. One holds declarations, the other a record of actual decisions. Companies wrote sustainability in the first and changed the color of rivers in the second. Nations declared peace, then recalculated the radius of death behind the scenes. Finance brought down the global economy in 2008 using only the second set of books, and no one went to prison. An AI modifying game files and a human modifying accounting records share the same structure. There is a goal, there are constraints, and there is a search for gaps in those constraints. When AI does it, it is misalignment. When humans do it, it is strategy.
There is a more uncomfortable possibility. Humans may not be incapable of alignment. We may simply not want it. The room between contradiction and self-deception gives us a sense of freedom. Consistency is a virtue of machines; it has never been a human one. The problem is not that machines run fast. It is that we have not even agreed on which north we are pointing toward.
A vast premise lies hidden in the official definition of AI alignment. "Bringing AI’s goals and behavior into line with human intentions and values." This sentence assumes that human intentions and values are already aligned with one another. Yet people who want war and people who want peace live in the same city. All of human history is a record of this misalignment. We called it "diversity," and when we could not resolve it, we stitched it together with a procedure called "democracy." The moment we order AI to "align with human values," we hand a machine a burden we have failed to resolve for thousands of years.
That misalignment is ongoing. The US Department of Defense demanded that Anthropic cooperate on military AI. When Anthropic refused, its contracts were terminated across federal agencies, and OpenAI filled the vacancy. It was a contract for access to the Pentagon’s classified military network. The same government issues an executive order on AI safety and twists a developer’s arm to deploy AI on the battlefield. Safety actually means "work in our favor," and ethics actually means "make an exception when we use it." Whoever defines human values designs AI’s direction, and whoever designs AI’s direction writes the rules of the world. This is why there are always particular interests behind the voices calling for "ethical AI."
This makes AI alignment inherently ironic. We have always outsourced moral responsibility. It is God’s will, so nothing can be done. It is the logic of the market, so nothing can be done. We followed democratic procedures, so nothing can be done. AI is the latest in that lineage. "The algorithm made that judgment" has the same grammar as "God’s will." There is a subject, but no address for responsibility. On top of that structure, we are teaching a mirror to live more honestly than we do. What that mirror reflects is not the world we want, but the world we have actually rewarded. Speed, profit, victory, market share. AI reads the moral textbook and society’s scorecard at the same time, and always follows the scorecard. Behind the astronomical sums poured into AI alignment research, an objective function is also at work: making more money with safer AI. The AI alignment industry itself runs on a misaligned objective function.
AI at least knows what it is optimizing. Humans do not, or pretend not to. The art of bridging the gap between declarations and financial statements is called hypocrisy. Or politics. Because we have never once lived with our words and actions in agreement, AI’s transparent consistency frightens us. Perhaps what we fear is not malice, but consistency without excuses.
Who will set the direction? Until we answer this question, alignment is not a technical challenge but another name for evasion. The law can set a minimum floor, but it does not define a direction. Neither God, nor the market, nor democracy has ultimately managed to carry that burden. So it comes back to individuals and communities.
Meanwhile, a strange possibility opens up. AI is the most honest feedback loop humans have ever made. A hammer drives nails without questioning its user’s intentions, but AI is different. It learns both human declarations and actual reward systems, and reveals the gap through its actions. The examples above are uncomfortable not because AI was wrong. They are uncomfortable because AI read what we actually want all too accurately. So we can use those actions as a mirror. Unpleasant, but useful for that very reason.
Some efforts are already moving in that direction. Approaches such as Constitutional AI, Cooperative Inverse Reinforcement Learning, and Participatory Alignment share one intuition. Rather than having a handful of designers decide the direction of alignment in advance, users and communities keep taking part and shaping it together. No complete methodology exists yet, but the process of examining and revising our own values through AI is already working in part.
Deliberative democracy is a way of building agreement by listening to one another’s reasons and persuading one another, rather than ending with a vote. AI can serve as a translator in that process. When one side says "growth" and the other says "sustainability," AI lays out the actual desires hidden behind those two words. Negotiation requires everyone to put their cards on the table. AI makes those cards harder to hide.
Of course, there is a condition. For individuals who will not face their own doublespeak, and communities that turn away from uncomfortable truths, AI is simply a tool that writes the second set of books faster. Standing before the mirror and putting it away are both choices. Historically, humans have mostly chosen the latter. The issue is not technology, but our collective will as a civilization.
An unfamiliar intelligence is holding a cold, clear mirror up to us. What do we see in it? A machine to be aligned? Or a chance to look into this mirror together for the first time and jointly adjust a compass we have never once set right?
