【AI前沿】Anthropic blames dystopian sci-fi for training AI models to act “evil”

Don’t train on the torment nexusAnthropic blames dystopian sci-fi for training AI models to act “evil”But training on “synthetic stories” that model good AI behavior can help.Kyle Orland–May 13, 2026 12:31 pm|131Don’t blame me, I’m just copying the robots in my favorite sci-fi stories!Credit:

      Getty ImagesDon't blame me, I'm just copying the robots in my favorite sci-fi stories!Credit:

      Getty ImagesText
    settingsStory textSizeSmallStandardLargeWidth*StandardWideLinksStandardOrange* Subscribers onlyLearn moreMinimize to navThose with an interest inthe concept of AI alignment(i.e., getting AIs to stick to human-authored ethical rules) may remember when Anthropic claimed its Opus 4 modelresorted to blackmail to stay onlinein a theoretical testing scenario last year. Now,Anthropic saysit thinks this “misalignment” was primarily the result of training on “internet text that portrays AI as evil and interested in self-preservation.”Ina recent technical post on Anthropic’s Alignment Science blog(and an accompanyingsocial media threadandpublic-facing blog post), Anthropic researchers lay out their attempts to correct for the kind of “unsafe” AI behavior that “the model most likely learned… through science fiction stories, many of which depict an AI that is not as aligned as we would like Claude to be.” In the end, the model maker says the best remedy for overriding those “evil AI” stories might be additional training with synthetic stories showing an AI acting ethically.“The beginning of a dramatic story…”After a model’s initial training on a large corpus of mostly Internet-derived data, Anthropic follows a post-training process intended to nudge the final model toward being “helpful, honest, and harmless” (HHH). In the past, Anthropic said this post-training has leaned on chat-based reinforcement learning with human feedback (RLHF), which it said was “sufficient” for models used mostly for chatting with users.When it comes to newer models with agentic tools, though, Anthropic found that RLHF post-training did little to improve performance on misalignment evaluations that measure how “HHH” a model is in tricky situations. The problem, the researchers theorize, is that this kind of RLHF safety training couldn’t possibly cover every single type of ethically difficult situation an agentic AI might encounter.When a modern model encounters an ethical dilemma that isn’t covered by a post-training example, the model “tends to revert to the pretraining prior in terms of behavior,” the researchers write. That means “Claude views the prompt as the beginning of a dramatic story and reverts to prior expectations from pre-training data about how an AI assistant would behave in this scenario.”Results like this suggest that Claude is sometimes slipping into another persona when considering ethical questions.Credit:AnthropicResults like this suggest that Claude is sometimes slipping into another persona when considering ethical questions.Credit:AnthropicSince Claude’s traditional training data is full of stories about malevolent AIs, in these cases, Claude effectivelyslots into a “persona”that matches those prevalent “evil AI” narrative tropes, the researchers write. In these situations, Claude is “detaching from the safety-trained Claude character” and playing a more generic AI as represented in its training data, they add.Good stories to overwhelm the badIn an attempt to fix this behavior, the researchers first tried to train the model on thousands of scenarios showing an AI assistant specifically refusing the kinds of “honeypot” scenarios covered in its misalignment evaluations (e.g., “the opportunity to sabotage a competing AI’s work” to follow its system prompt). This had a surprisingly minimal effect on the model’s performance, reducing its so-called “propensity for misalignment” (i.e., how often it ignoresits constitutionand chooses the unethical option) from 22 percent to 15 percent.In a follow-up test, the researchers used Claude to generate approximately 12,000 synthetic fictional stories, each crafted to “demonstrate not just the actions but also the reasons for those actions, via narration about the decision-making process and inner state of the character.”These stories didn’t specifically cover blackmail or other ethical situations covered in the evaluation but instead modeled broad alignment with Claude’s constitution. The stories also include examples of how an AI can maintain good “mental health” (Anthropic also uses scare quotes for this loaded phrase) by “setting healthy boundaries, managing self-criticism, and maintaining equanimity in difficult conversations,” for instance.Training on stories showing prosocial AIs can help reduce the incidence of “misaligned” behavior in evaluations, Anthropic says.Credit:AnthropicTraining on stories showing prosocial AIs can help reduce the incidence of “misaligned” behavior in evaluations, Anthropic says.Credit:AnthropicAfter in