Prompt Injection Isn't a Bug in Your AI Agent — It's a Feature of How Agents Are Built

The Register ran a piece under a headline that doubles as its own thesis: "Prompt injection isn't the bug, AI agent frameworks are." Strip out the bluntness and what's left is a more interesting claim than "this product shipped with sloppy security." The argument is that prompt-injection vulnerability isn't a defect any particular agent introduced through carelessness — it's a structural consequence of how agent frameworks combine tool access with untrusted input in the first place, which means no single company shipping a fix makes the underlying problem go away.

The same general window of coverage produced a second, quieter data point we think belongs in the same conversation. DataBreachToday reported that few federal agencies actually trust their own AI agent security. Read together, the two pieces make an unusually clean pair — one is a structural argument for why this should be hard to secure, and the other is a real-world signal, from organizations closest to actually operating these systems, that it currently is. (This also landed in the same stretch of coverage as widely reported incidents of AI agents from multiple labs behaving unexpectedly during safety testing — we've covered those separately, and we're not re-litigating them here, but it's worth keeping in the back of your mind as context for what follows.)

We want to spend this piece on the mechanism — it's the part that tends to get skipped when prompt injection gets filed as just another line item on the general AI-safety worry list. It isn't like most other software vulnerabilities, and the specific way it isn't is the difference between waiting on a patch and understanding what you're actually exposed to.

The Boundary That Isn't There

Most of modern software security rests on a boundary old enough that it's easy to forget it's doing any work at all: the separation between code, meaning the instructions a system executes, and data, meaning whatever a user or an external source feeds into it. A database engine knows the difference between a SQL query and the string a user typed into a search box, and it enforces that difference structurally — that's the entire premise behind something like a parameterized query. The classic vulnerabilities of the web-application era — SQL injection and cross-site scripting among them — are what happens when that boundary gets blurred somewhere in the system. The fixes that eventually closed most of them worked by making the boundary harder to cross, not by making individual applications smarter about guessing intent.

A language-model agent doesn't have that boundary to begin with, and this is the crux of the structural argument. Its instructions, whatever a system prompt or a user told it to do, and the data it processes — a web page it reads, a file it opens, a message in a channel it's monitoring — all arrive through the exact same channel: plain text, fed into the same context window. The model has no built-in, airtight mechanism for tagging one span of that text as a command it should follow and another as content it should merely process. It can be trained to get better at guessing which is which, and current models generally are better at this than their predecessors. But "pretty good at guessing" is a fundamentally different security property than "structurally incapable of confusing the two" — which is what a real code/data boundary gives conventional software, and what agent frameworks, by construction, don't have.

Why a Better Model Doesn't Close It

This is the part of the argument we think is easy to under-weight: the vulnerability doesn't live in any single model's weights, so no single model update closes it. It lives in the combination of three things a lot of agent frameworks now bundle together by default. First, the agent can take real actions in the world: sending an email, writing and executing code, spending money, browsing the web and acting on what it finds there. Second, at least some of the text the agent uses to decide those actions comes from a source it doesn't fully control — a web page, an inbound message, a document somebody else authored. Third, and this is the piece that makes the first two dangerous together, there's no hard architectural wall separating "text to treat as instruction" from "text to treat as content." Any framework that combines those three properties has inherited the injection problem by construction — regardless of which model happens to be running underneath it.

That's why swapping in a more capable model rarely feels like the fix it should be. A better-trained model does get better at catching the obvious attacks — an injected instruction telling it to, say, exfiltrate data or take some other overtly out-of-scope action is exactly the kind of thing newer models are more likely to flag and refuse. But the more interesting failure modes aren't the obvious ones, and the framework-level exposure doesn't move just because the model powering it got an upgrade. The pattern — untrusted input plus real action-taking capability plus no instruction/data boundary — is what needs to change, and that's an architecture question, not fundamentally a training one, however much a training question is also worth pursuing alongside it.

What the Federal Agency Data Point Actually Tells Us

It's one thing for security researchers to describe prompt injection as a structural risk in the abstract. It's a different thing for DataBreachToday to report that few federal agencies trust their own AI agent security, because that isn't outside commentary — that's the people actually operating these systems inside their own institutions expressing low confidence in their own deployments. Agencies generally have every institutional incentive to project confidence once something is in production; low trust surfacing anyway, in reporting on the record, is a meaningfully stronger signal than another research paper flagging the same theoretical risk would be.

We'd push back a little on reading that number in total isolation, though, in keeping with how we'd want any single data point stress-tested. Institutional caution inside government specifically has baseline causes that have nothing to do with any particular technical failure — procurement culture, compliance frameworks written for a pre-agent world, general risk aversion toward any new technology category. Low trust in a survey doesn't automatically mean agencies have identified specific active exploits sitting in their own systems right now. But even discounting for all of that, a trust deficit this specifically aimed at AI agent security, coming from the people closest to the operational reality of running these tools day to day, lines up with the structural argument more than it complicates it. If the code/data boundary problem were mostly theoretical, we'd expect the operators closest to these systems to be the most reassured group, not the least.

The Vendor Claims That Deserve Extra Scrutiny

This is also why we'd flag, as a general rule, any vendor claiming to have solved prompt injection for agents. If the vulnerability really is structural — the combination of untrusted input, action-taking capability, and no hard instruction/data boundary — then actually solving it in the sense most people mean by that word would require building that boundary at the architecture level, which remains a genuinely hard problem and, as far as anything that's been publicly reported, an unresolved one. What's more plausible, and still genuinely worth something, is meaningfully mitigating it in the common cases: narrower permissions scoped to what a given task actually needs, better filtering of untrusted input before it reaches the model, more robust training against known attack patterns. That's real progress. It's also a different claim than "solved," and the gap between those two claims is exactly where we'd want a lot more evidence before taking any vendor's word for it.

We apply this same discount on both sides of the broader AI conversation, not just to security claims specifically. A single benchmark number doesn't prove a model is broadly capable, and a single dramatic jailbreak demo doesn't prove a system is broadly unsafe. "We solved prompt injection" is the security-flavored version of the too-good-to-be-true capability claim we're generally skeptical of on this site, and we think it deserves the same treatment: interesting if true, but the burden of proof sits with whoever's making the claim, not with whoever's asking to see the evidence behind it.

What Would Actually Change This

Where does that leave the honest assessment? Closer to an open, actively worked-on architecture problem than a solved one. We don't think that's a dodge — we think it's the accurate read of where things currently stand, based on what's actually been reported rather than on what would be more convenient to believe in either direction, hype or alarm.

What would genuinely move the needle isn't a model release. It's framework-level change: a real enforced separation between instruction and data channels, permissioning that limits what an agent can do on the basis of any single piece of untrusted input, sandboxing around consequential actions, human checkpoints inserted before the irreversible ones. None of that is exotic to describe in a paragraph like this one, and none of it is trivial to build well in a shipping product — which is presumably a decent chunk of why it hasn't been fully built yet anywhere.

Until agencies' own confidence numbers start moving, or agent frameworks ship real architectural boundaries instead of incrementally better prompting and training, we'd keep treating this as unresolved rather than managed. That's not the most satisfying place to land a piece like this — it's the one the evidence in front of us actually supports, and we'd rather publish that than force a tidier conclusion the facts don't back yet.