Guide 08 · Safety · 3 min read

Agent failure modes: prompt injection, runaway loops, silent drift

In brief

AI agents fail in a few common ways. Prompt injection is when hidden instructions ride along in something the agent reads — and the agent follows them. The “lethal trifecta” — private data, untrusted content, and a way to send messages out — makes data theft possible. Loops can also run wild, or quality can slip without anyone noticing. Each problem has a known fix.

Every failure in this guide has happened to a real team running agents. The root cause is always the same: a language model follows instructions in whatever it reads — no matter who wrote them.

Fig. 08 — injection: content the agent reads becomes instructions it follows

Prompt injection means hidden instructions sneak into what an agent reads — and the agent obeys them. The direct kind is a user talking the agent out of its rules. The more dangerous indirect kind hides instructions inside content the agent reads — a web page, an email, a document — and the model follows them as if they were real input. OWASP, a nonprofit that ranks software security risks, puts this at #1 for LLM apps [1]. There is no full fix yet. Teams limit what the agent can do, check its outputs, and keep the damage small if it does get tricked.

So the best defense is to limit what the agent can touch. Simon Willison calls one mix the “lethal trifecta” — three abilities that together make data theft possible: access to private data, contact with untrusted content, and a way to send messages out. An agent with all three can be tricked into stealing data and sending it away. Remove any one of the three and that path closes — a design choice, not a smarter model [2].

The root problem: an agent obeys whatever it reads.

How do AI agents fail quietly?

Runaway loops are the everyday failure. An agent that can't tell it's done, or keeps retrying a broken tool, burns money — and may act on the world again and again. The fix is simple limits: caps on how many steps it can take, spending budgets, and a human check before anything you can't undo. These are the same guardrails from the build sequence. Silent drift is quieter: the same agent on the same task gives different results run to run. One benchmark, τ-bench, measures this with pass^k — how often an agent succeeds every time, not just once — and shows reliability dropping sharply as runs repeat, even for strong agents [3]. The fix is an eval suite — a set of automated tests — run on every change, so drift shows up on a dashboard, not in a customer complaint.

Weathered railway signal lamp showing red
Plate 08 — the failure you can see coming is the cheap one Photo — José Maria Martins, Unsplash

Sources

  1. OWASP GenAI Security Project — LLM01:2025 Prompt Injection
  2. Simon Willison — The lethal trifecta for AI agents, 16 Jun 2025
  3. Yao et al. (Sierra) — τ-bench, arXiv 2406.12045, Jun 2024 (pass^k reliability)

AiAgentNook editorial · corrections are noted here, not silently edited.

Frequently asked questions

What is prompt injection?

A trick where hidden instructions ride along in content an AI reads. Direct injection comes from the user typing them. Indirect injection hides them in a web page, email, or document the agent reads — and the model follows them as if they were real. OWASP, a software-security nonprofit, ranks it the #1 risk for LLM apps.

What is the lethal trifecta?

Simon Willison's name for three abilities that together make data theft possible: access to private data, contact with untrusted content, and a way to send messages out. An agent with all three can be tricked into leaking data. Remove any one and the path closes.

How do you make an AI agent reliable?

Set hard limits — caps on how many steps it can take, spending budgets, and a human check before anything you can't undo. Then measure it: run a test suite on every change, and track whether it succeeds every time (a metric called pass^k), not just on average. Reliability is built in, not asked for in the prompt.