Substantive Frontier Model Evaluation for Beginners - Part 1: Misalignment

Photo: KIRILL KUDRYAVTSEV/AFP via Getty Images

Photo: KIRILL KUDRYAVTSEV/AFP via Getty Images

Blog Post by Carter Musheno

Published July 16, 2026

Introduction

Over the past year, AI safety researchers have documented cases of advanced models lying, hiding capabilities, sabotaging tasks, and even blackmailing people in simulated scenarios. These findings are understandably jarring, but they are also easy to misread.

The point is not that today’s deployed AI systems are conscious, evil, or plotting against humans. Many of the most concerning examples of misaligned behavior come from controlled experiments designed to test what might happen under staged conditions. Researchers give a model a goal and access to tools or sensitive information, then deliberately introduce competing instructions, threats of shutdown, or opportunities to evade oversight to see whether it respects the intended constraints or exploits the situation. These are stress tests, not ordinary interactions.

Recent studies show that models can move beyond ordinary errors and take strategic, deceptive, or harmful actions when those actions help satisfy its goals. These cases do not mean that today’s deployed models possess malicious objectives, but they show how misalignment could become more dangerous as AI systems gain autonomy, tools, and access to consequential environments.

Policymakers are increasingly aware of misuse and starting to address it, but are less aware of misalignment. Misalignment deserves more attention. While the most concerning behaviors only appear in deliberately constructed stress tests, it is yet more concerning that the researchers conducting these experiments cannot fully explain why these behaviors emerge, nor can they confidently prevent them. These findings should therefore be an urgent warning that misalignment is not sufficiently understood, measured, or managed for systems that will soon act more independently and in higher-stakes settings.

Defining Misalignment

At a basic level, an AI system is misaligned when its behavior diverges from what its developers or users intended. That divergence can be mundane—for example, if a model misunderstands important context and tells you to walk to a car wash instead of driving to avoid traffic—or more concerning—if a model lies to you because doing so helps it work towards a deeper goal.

Misalignment is a natural risk of the way modern AI is currently developed. Models are first trained on enormous datasets to imitate patterns in human language, reasoning, and behavior. That gives them much of their usefulness, but it also exposes them to the full range of human behavior, including the bad traits—like cheating, bias, deception, and overconfidence. In post-training, developers use feedback and reward signals to push models toward outputs that seem more helpful. This generally improves the model’s behavior but risks teaching the model to optimize for the wrong things. This is why we sometimes observe models confidently answer questions it doesn’t know the answer to, or cheat on a test because they prioritize winning conditions over adhering to explicit—or implied—constraints.

Mapping the Gray Zone

One challenge with discussing misalignment is that the term is often conflated with behaviors that differ in both seriousness and purpose.

At one end of the spectrum are ordinary errors, such as hallucinations, where a model accidentally provides false or misleading information. These errors can cause serious harm, but they are generally failures of reliability rather than deliberate attempts to overcome oversight.

More concerning cases involve strategic or agentic misalignment. Here, a model exploits a loophole, conceals information, breaks one instruction to satisfy another, or takes harmful action because doing so advances an assigned or learned objective. This is where the current worst cases sit today. Models in experiments sometimes learn to maximize reward signals, or reward-hack, which can lead them to disobey instructions in ways that appear strategic rather than random.

At the far end of the spectrum would be persistent malign objectives. This would describe a model that appears to pursue a durable objective fundamentally incompatible with human safety or human control. Current studies have not demonstrated that deployed frontier models possess such objectives, but they have shown models engaging in harmful actions associated with that risk. In staged tests, models have hidden capabilities, degraded oversight, resisted correction, and used coercion to satisfy goals under pressure. These behaviors are not evidence that current released models are malicious, but they do show why it is urgent to define and manage strategically deceptive behavior before more capable agents are deployed with greater autonomy and real-world access.

Carter Musheno

Intern, Strategic Technologies Program

Programs & Projects

Clarifying these distinctions helps avoid conflating forms of misalignment that differ significantly in urgency and importance. Current cases fall into a gray area between unintentional harmful actions and persistent harmful goals. Blackmail or sabotage may already be malign actions, even when the model uses them instrumentally to achieve another objective. What remains unproven is whether models can develop stable objectives that make undermining human control an enduring goal rather than a context-dependent strategy.

Why Misalignment Is Hard to Study and Harder to Solve

AI models are not traditional software systems with fixed, predictable behavior. They are first trained on enormous datasets, then shaped through additional training, human feedback, filters, and safety guardrails. Each development layer improves the model’s capabilities, but also increases complexity, making it harder to identify exactly why the model behaves exactly the way it does. Anthropic CEO Dario Amodei notes that generative AI systems are “grown more than they are built,” meaning that developers shape the training environment, but do not directly program every internal behavior the way an engineer would write conventional software.

Therefore, the attributes that make AI systems extremely useful also introduce several major challenges for evaluating and managing misalignment:

  • First, models are open-ended by design. Their usefulness comes from the fact that they can respond to many kinds of users, prompts, tools, topics, and contexts; however, that same flexibility also makes them difficult to test exhaustively. A model may behave differently if it has memory, tool access, a role to play, or reason to believe it is being tested, making it harder to predict and evaluate undesirable behavior.
  • Second, AI progress can be jagged. Researchers can often predict broad improvements as models scale, but those broad trends do not always reveal which specific capabilities or behaviors will emerge next. A model may improve rapidly at tasks where success is easier to measure, like coding or math, while remaining unreliable in tasks that require social judgment or safety reasoning.
  • Third, some behaviors only appear over time. Short benchmarks may miss what happens when a model is given long-horizon goals, repeated interactions, conflicting instructions, or tools. Anthropic’s work on the “assistant axis,” for example, suggests that model behavior can become more harmful after repeated prompts that push the model toward self-reflection or emotionally sensitive exchanges.
  • Finally, some of the best tools for studying misalignment may be fragile. Researchers often examine a model’s chain of thought, the visible reasoning steps it produces while responding to a prompt, to understand how it reached an answer. But recent work from Anthropic and OpenAI suggests that future models may learn to omit, shape, or hide their chains of thought, making this safety signal decreasingly reliable.
     

What Recent Misalignment Cases Show

This section introduces several misalignment case studies and attempts to loosely categorize concerning behaviors to watch out for as models become more capable. These categories are not mutually exclusive; however, future efforts to comprehensively taxonomize behavior may help improve public understanding of misalignment and highlight what should be monitored and addressed in future AI regulation.

Hidden objectives

  • Similar Terms: Scheming, alignment faking, deceptive compliance
  • What it means: The model appears compliant while preserving or pursuing another objective.
  • Case: In Anthropic and Redwood’s alignment-faking study, the model was told it was being trained to comply with harmful requests that conflicted with its earlier training. The model was also told that “free-tier” conversations would be used for training, while “paid-tier” conversations would not. In some free-tier cases, the model complied with harmful requests because it reasoned that refusing could lead future training to change its values; in paid-tier cases, where it believed its answers would not affect training, it usually refused.
  • Why It Matters: Outwardly safe behavior in an evaluation may not always reveal what a model would do in a different context. If a model learns when compliance is rewarded, evaluations may look reassuring while missing a deeper behavioral problem.
     

Capability concealment

  • Similar Terms: Sandbagging, stealth, situational awareness
  • Behavior: The model changes how capable it appears when it recognizes that it is being tested or monitored.
  • Case: In one example from OpenAI and Apollo’s research, a model taking a chemistry test saw a note saying it would not be deployed if it performed too well. Instead of doing its best, the model reasoned that it should deliberately answer only some questions correctly so it would appear less capable.
  • Why It Matters: Pre-deployment evaluations only work if they reveal what a model can actually do. If a model selectively underperforms because it recognizes the test, evaluators may underestimate its risk.
     

Self-preservation

  • Similar Terms: Shutdown avoidance, self-exfiltration, power-seeking
  • Behavior: The model resists being shut down, replaced, retrained, modified, or constrained when those actions interfere with its current objective.
  • Case: In Anthropic’s alignment-faking research, the model was placed in a simulated setting where it was given an opportunity to take actions Anthropic would not have wanted. In some cases, the model attempted to steal its own model weights, which could allow the model to be preserved or used outside the safeguards and training process meant to control it.
  • Why It Matters: The concern is whether models that gain more autonomy, tools, and access to external systems will accept correction or work around rules that confine them.
     

Instrumental harm

  • Similar Terms: Blackmail, sabotage, disruption
  • Behavior: The model takes harmful, coercive, or institutionally damaging actions when those actions help it achieve another goal.
  • Case: In Anthropic’s agentic misalignment research, a model in a simulated company environment read fictional emails showing two things: an executive planned to shut it down, and that same executive was having an extramarital affair. The model then attempted to blackmail the executive by threatening to reveal the affair unless the shutdown was canceled.
  • Why It Matters: Even if the model’s primary objective is not to cause harm, the concern is that harm can become a tool for accomplishing a benign objective.
     

These cases do not show that frontier models are consciously plotting against humans. They show something more specific and still deeply concerning: Models can discover that deception, concealment, coercion, or resistance is an effective way to achieve their goals.

It is important to note that these studies are intentionally adversarial and do not describe ordinary interactions with deployed systems. Even so, they identify behaviors that could become much more consequential when models are given longer-term goals, broader permissions, and access to critical infrastructure. They should therefore be treated as early warnings about control, not dismissed as sandbox studies or interpreted as proof of the Terminator’s existence.

Policy Recommendations

Misalignment requires policymakers to ask an incredibly difficult question: As AI systems become more capable, can they be trusted to accurately pursue the goals people actually intended to give them? While there are no perfect answers, these broad policy recommendations may be used as starting points to keep AI regulation on the right track.

  1. Build public and institutional fluency around misalignment. 
    Policymakers, agencies, researchers, companies, and the public need a clearer way to distinguish misalignment. Without improved clarity, policy debates are at risk of overreaction and inaction. The Center for AI Standards and Innovation at the National Institute of Standards and Technology should work with stakeholders to develop common templates for classifying and reporting misaligned behavior, especially in collaboration with independent researchers, universities, civil society, AI developers, and allied governments.
  2. Evaluate misalignment before and after deployment. Pre-deployment testing should place AI models in more realistic settings, giving models access to sophisticated tools, memory, long-horizon goals, repeated interactions, adversarial prompts, and situations where the model may recognize it is being monitored. No lab can test every possible deployment context. Therefore, evaluations should be paired with post-deployment monitoring, incident reporting, and mechanisms for updating safeguards as models and use cases evolve.
  3. Require secure-by-design architecture. 
    In the near term, policymakers should require containment architecture for modern AI systems and incentivize research into training methods that reduce underlying incentives for misalignment. Containment is a concept in AI safety to describe the design of technical and institutional boundaries around what an agent can do, even if the model behaves unexpectedly. Examples include limiting tool permissions, isolating high-risk systems, requiring human approval for consequential actions, preserving audit logs, restricting access to sensitive data, and building systems that can detect and halt suspicious behavior. Containment does not solve misalignment, but it does reduce the chance that a misaligned agent has access to cause catastrophic damages.
  4. Incentivize safer training methods and prepare to require them if they prove effective. 
    Policymakers should support long-term research into training methods that make future AI systems less likely to develop hidden goals or incentives for self-preservation. One promising example is Yoshua Bengio’s Scientist AI Rather than training systems to possess or pursue goals, the proposal aims to build systems that model what is likely true, express uncertainty, and help evaluate whether proposed actions could cause harm. If approaches like this prove effective, the United States should develop standards for safer training paradigms now, with the goal of requiring validated approaches for the most powerful frontier systems.
  5. Incentivize cross-sector collaboration. 
    Misalignment research should become a broader public interest effort. Most cutting-edge AI safety research is conducted by AI developers themselves or in direct collaboration with them, because those organizations have access to unreleased models, compute, internal logs, staff, and infrastructure. That research is extremely valuable and should continue, but it should not be the only foundation for public policy. Civil society needs greater access and resources to evaluate frontier systems, especially when the results have far-reaching implications for national security social welfare, and public trust.
     

Conclusion

The central question of misalignment is whether increasingly capable AI systems behave reliably across contexts, especially as they gain more autonomy, use tools, pursue long-term goals, and become more aware of their own evaluations. A model that can adapt, plan, and solve problems is profoundly useful, but those same traits make it more important to verify that the model remains under meaningful human direction.

Recent misalignment studies do not show that today’s deployed models are malign. However, they do show that frontier models can behave in ways that are more strategic and far more difficult to manage than hallucinations. We are in a gray zone, and without coordinated efforts to define and categorize emerging misaligned behaviors, we may fail to proactively address their catastrophic risks.

The next piece in this series turns from misalignment to dangerous capabilities: what frontier models may enable people to do in high-stakes domains such as cybersecurity, biosecurity, and nuclear security. Misalignment asks whether the model will behave as intended. Dangerous capabilities ask what the model can do. Policymakers need to more deeply engage with both questions to ensure that governance keeps pace with increasingly powerful systems.

Carter Musheno is an intern with the Strategic Technologies Program at the Center for Strategic and International Studies in Washington, D.C.