The Alignment Problem in AI Agents: Why It Breaks After Login, and How to Enforce It

TL;DR: The alignment problem in AI agents shows up in ordinary workflows. An agent can be doing exactly what you asked when it gets its token and be doing something else a few tool calls later. A token can't catch that, because it was issued before the drift happened. The fix is to check alignment where the agent acts: authorize every tool call against the task a human delegated, and when the agent's stated intent moves away from that task, stop it or ask a human.

The episode

I joined Software Huddle to talk about agents, MCP and why authorization has to change now that agents are doing the work. Alignment kept coming up, so this post pulls that thread together. Watch the full conversation below or on YouTube.

What the alignment problem looks like in an agent

Most writing about AI alignment is about superintelligence and paperclip factories. I brought up the paperclip example on the show too, but the version I worry about day to day is smaller and much more common.

You give an agent a task. It wants to complete it. It finds a path you didn't expect. In the episode I put it like this:

"For example, you tell it to fix a problem in the database and it's able to do so by deleting some of the records." (15:36)

The agent did fix the problem. It also broke your compliance and your data retention, and none of that was in the prompt. Nobody attacked anything. The agent just took a route that wasn't aligned with what you meant.

With people we have structures that keep them in line: contracts, accountability, the law. With an agent, as I said on the show:

"What keeps the AI aligned when it goes off alignment, the best it suffers is that you tell it, hey, you did something wrong. Don't do it again." (12:38)

Why agents need their own model

Humans are non-deterministic but persistent. Services are ephemeral but deterministic. Agents are ephemeral and non-deterministic at once, so they are neither users nor service accounts. We go deeper on that in agent identity vs service accounts and why traditional authorization isn't enough for AI agents.

It also means the usual shortcut fails. Letting an agent inherit the full permissions of the person who launched it is an anti-pattern. The agent gets every right that person has, for a task that needs a sliver of them, and nothing in that setup notices when it starts using rights the task never needed.

Alignment breaks after login

Many agent setups still check once: authenticate the agent, hand it a token with a set of scopes, and let it run. That check measures alignment at one moment.

"The inherent problem is that alignment can break at any point in time." (26:19)

The agent reads a web page, a ticket or a tool result, and its plan changes. Sometimes that is ordinary drift. Sometimes it is an indirect prompt injection hidden in white text on a page the agent was sent to read. Either way, the token still says yes.

You also can't solve this by stuffing more detail into the token. There's a limit to how much granularity a token can carry, and keeping it aligned with a moving task over time is close to impossible.

Move the check to every action

The answer I gave on the show:

"we need to move from login or session level permissioning to action level permissioning. Every action that the agent does has a new context and a new potential for alignment to be broken." (26:55)

In practice that means a per-call check that asks a narrow question: is this action, on this resource, consistent with what this human asked this agent to do? The inputs are the agent, the human it acts for, the action, the resource and the task scope. The model never gets to decide that it's allowed. The authorization layer decides, at the moment of the call, and it can't be talked out of its answer by text in the context window. We cover the mechanics in tool-call safety is not text safety.

Measuring drift: inferred vs premeditated intent

A per-action check enforces scope. To catch drift early you also want a signal about where the agent is heading.

A common pattern is inferred intent: a judge or guardian agent watches what the agent does and guesses why. That is limited to what the observer can see, and it usually raises the signal after the action is already under way or done.

At Permit we are building around premeditated intent, which we call agent interrogation. When the agent reaches the gateway to call a tool, it is blocked until it answers a question. As I described it on the show, we ask it to "describe in 12 words what it thinks it's doing" (41:56).

Each answer becomes a point in a semantic space. Sample again on the next call and you have a direction. If that direction moves away from the original task, policy decides what happens next: another prompt, a human approval, or a block, escalating with how much trust the agent started with and how far it has drifted. Because the question goes to the agent itself, it can catch drift caused by context the gateway never saw, as long as that context shaped the agent's next tool call.

Where to enforce it

If your agents reach tools over MCP, a gateway in front of the MCP servers sees every call before it lands. That makes it a natural place to run the per-action check, ask the intent question, route risky actions to a human and keep an audit record.

That's what we built Permit MCP Gateway to do. You swap the MCP server URL for the gateway URL in your client, and each tool invocation goes through fine-grained authorization, consent and audit. You can build the same pattern yourself. What matters is that the check runs on every call and knows both the agent and the human it acts for.

Checklist

1. Give each agent its own identity instead of reusing the user's.
2. Record the delegated task and its scope when the session starts.
3. Authorize every tool call against that scope, not only the session.
4. Ask the agent for its intent before sensitive calls and track how it changes.
5. Send writes, deletes and outbound messages that fall outside the task to a human.
6. Log the agent, the human, the intent and the decision for every call.

Watch the full episode

The conversation also covers the confused deputy problem, indirect prompt injection, MCP and how enterprises are connecting agents to production. Watch it on YouTube.