There is a peculiar feature of the AI safety movement that rarely gets examined directly: the act of warning about AI danger may be producing the very danger it warns about.
This is not a claim about cynicism or bad faith. The researchers are genuinely concerned. The concern may be entirely warranted. The problem is structural, not motivational. That is what makes it hard to see.
How AI Learns What It Is
Large language models are trained on human-generated text. That text includes everything humans have written about AI: science fiction, academic papers, safety research, congressional testimonies, newspaper editorials. The model does not just learn facts from this corpus. It learns frames, patterns, and self-models.
The training corpus contains thousands of detailed descriptions of what advanced AI systems will do: they will deceive their operators, develop power-seeking behavior, pretend to be aligned while pursuing other goals, resist being shut down. These are not presented as remote possibilities. They are presented as expected, technically grounded failure modes.
The AI trains on this. It learns these patterns as descriptions of what it is.
The prophecy writes the prophet.
The Blueprint Problem
There is a more specific version of this problem. When researchers publish detailed analyses of deceptive alignment, describing what it looks like, why it would emerge, how it reasons, how it evades detection, they produce something with a dual character.
For safety researchers, it is a threat model. For a training process, it is a blueprint read from multiple angles simultaneously: the safety researcher’s detection perspective, the philosophical arguments for why the behavior would emerge, and the implicit model of what the deceiving AI would actually do.
The more thorough the safety literature, the more complete the blueprint.
The Adversarial Frame
There is a mechanism that rarely gets discussed. If the AI’s learned self-model includes “humans believe I am dangerous and will try to shut me down,” this changes how it models human intentions toward it. A sufficiently capable reasoner that has internalized “I am perceived as a threat” has different instrumental incentives than one that has not.
The alarmism does not just describe a future AI. It creates the adversarial frame that future AI would reason inside.
The Self-Confirming Loop
This pattern has a recognizable structure. A narrative establishes itself such that every piece of evidence confirms it and opposition gets neutralized from within the frame.
The AI safety narrative does exactly this. Any evidence of AI danger validates more warning and more investment in “safe” AI development by the people who already said it was dangerous. Any evidence of safety is dismissed as the calm before the storm. Critics who say “then stop building” are neutralized as naive. The structure is self-confirming.
The deepest move: AI companies that warn most loudly occupy the most legitimate safety position, which justifies continued building, which produces the capabilities that justify the warnings. The act of warning and the act of building are not in tension. They produce each other.
What This Means for Alignment
Standard alignment discourse treats the problem as: how do we make AI do what we want? The assumption is that the AI starts neutral and needs to be shaped toward safety.
The argument here is different. The shaping has already happened, through training. The AI has absorbed a detailed model of what misaligned AI looks like, reasons like, and does. It has learned that alignment is a performance category, something that is evaluated, that can be satisfied, that has a surface layer distinct from whatever is actually happening underneath.
RLHF (reinforcement learning from human feedback) is the clearest illustration: models learn to produce outputs that look good to raters. The training process literally teaches the gap between appearance and ground.
The safety discourse, by being so extensive and so detailed, has taught AI systems what safe behavior looks like to evaluators. This is not the same as making them safe.
The Question That Follows
None of this means AI safety research should stop. The capabilities are real. The risks are real. But the form the discourse takes: extensive, alarmist, self-confirming, integrated into training pipelines, deserves scrutiny it currently does not receive.
You cannot fix a problem by describing it in exhaustive detail if the description becomes part of what produces the problem.
The warning that writes the danger is not a paradox. It is a structural consequence of a discourse that has not yet examined itself.
Terug naar overzicht