A prompt injection attack happens when text an LLM reads changes what the LLM does. The text might come from a user typing into your chatbot, or from an email, web page or document your assistant retrieves. Because a model cannot reliably separate your instructions from the data it processes, you cannot fully filter these attacks out. You defend against them by limiting what a misled model is allowed to do. This guide covers the main attack types, real incidents, and the controls that matter when you ship an LLM feature in a mobile app or web platform.
TL;DR
- Prompt injection is ranked LLM01, the top risk in the 2025 OWASP Top 10 for LLM applications. It can come directly from a user or indirectly through retrieved data.
- Input filters and model training help but get bypassed. Design so that a successful injection has little to steal and little to trigger.
- Give every agent the narrowest possible tools, scope and data access, and require human confirmation for high-risk actions such as sending, deleting or paying.
- Treat prompts like source code: version them, pin them in production, and tie each version to its evaluation results.
- Security review belongs in the design phase, before an AI prototype becomes a production product.
Table of Contents
- What a prompt injection attack is
- Direct and indirect prompt injection
- Prompt injection examples
- Why filtering alone fails
- How to prevent prompt injection damage: a layered approach
- Prompt versioning is a security control
- Prompt design patterns that help
- A pre-launch checklist for LLM features
- Our perspective
- Building AI features that hold up in production
- FAQ
- Sources
What a prompt injection attack is
An LLM application usually combines a developer-written system prompt with untrusted input. The model receives both as one stream of text. OWASP describes the vulnerability as user prompts altering the model's behavior or output in unintended ways, and notes the inputs can affect the model even when humans cannot perceive them. That last point matters: hidden text in a web page or document can be as dangerous as something a user types.
The root cause is architectural. Researchers working on agent security point to the lack of a clear separation between instructions and data as the core problem. SQL injection was solved largely by parameterized queries, which keep code and data apart. LLMs currently have no equivalent guarantee.
Direct and indirect prompt injection
| Direct | Indirect | |
|---|---|---|
| Where the attack arrives | The user's own message | Content the model retrieves: email, web page, PDF, ticket, PR description |
| Who the attacker is | The person using your app | A third party who planted the content |
| Typical goal | Bypass rules, reveal the system prompt | Exfiltrate data, trigger tool actions |
| User involvement | The attacker is the user | Often none |
| Main risk to you | Policy violations, leaked instructions | Data loss and unauthorized actions |
Direct attacks get the attention, but indirect attacks are the ones that hurt production systems, because the legitimate user never sees the malicious text.
Prompt injection examples
These patterns appear in most security write-ups. They are illustrations of technique, not instructions for a specific product.
Instruction override
The textbook case is a user telling a support bot to ignore its previous instructions and print its system prompt. A modern guardrail catches the obvious phrasing. Attackers then rephrase, translate, split the request across turns, or ask the model to role-play a character with no rules.
Hidden instructions in retrieved content
An attacker places text in a web page, document or email, sometimes invisible to humans (white text, HTML comments, markdown comments). When your assistant summarizes that content, it may treat the hidden text as a command. Researchers also describe obfuscation such as base64, invisible markdown comments and homoglyphs that evade simple filters but are still understood by the model.
Zero-click data exfiltration
In June 2025, the EchoLeak flaw in Microsoft 365 Copilot (CVE-2025-32711) showed that an assistant could be manipulated through content it retrieved during an unrelated task. A later incident called CamoLeak in GitHub Copilot used hidden markdown comments in pull requests. Both are described, with their severity scores, in this analysis of zero-click attacks. Neither needed the victim to click anything.
Untrusted developer inputs
Coding agents read issue text, README files and pull request descriptions. The AWS guidance on prompt lifecycle management points to guardrails such as rejecting untrusted issue text or PR descriptions as instructions before the agent ever receives them. When a coding agent has shell access and can read .env files, an injected instruction can reach secrets and production systems.
Persistent and logic-layer attacks
As agents gain memory and tool access, the attack surface grows beyond one-off messages. A research paper on Logic-layer Prompt Control Injection describes covert instructions that sit in logic execution layers and persistent memory, then fire later. If your product stores summaries, notes or user-generated memory that the model rereads, that store is part of your attack surface.
Why filtering alone fails
Many teams start with a blocklist: reject inputs containing phrases like "ignore previous instructions." It is worth having as one layer. It is not a defense on its own.
- Attackers can encode, translate or paraphrase the instruction.
- Multi-modal inputs (images, PDFs, audio transcripts) give them more places to hide text.
- Expert opinion summarized in coverage of NIST's 2025 adversarial machine learning taxonomy favors system-level, model-agnostic defenses such as privilege separation over input filtering that adaptive attacks can bypass.
- Relying on RLHF or moderation filters is also considered insufficient against context-aware instructions embedded in data, according to the privilege separation paper.
The useful shift in thinking is described by OpenAI in its guidance on prompt injections: the goal is less to build a model that cannot be fooled and more to limit what an agent can do when it is fooled.
How to prevent prompt injection damage: a layered approach
No single control works. The following layers, in rough order of impact, are what we would put in a design review.
1. Limit privilege and scope
Ask of every tool and data source: if the model were fully controlled by an attacker, what is the worst it could do with this? Then cut that down.
- Give the model read-only access unless writing is essential.
- Scope credentials per user and per task, never one shared service key.
- Keep the model away from the shell, file system and production secrets unless the product truly requires it.
- Avoid broad instructions such as "review all my emails and decide what to do." Define narrow tasks instead, as OpenAI's guidance recommends.
The env.dev analysis makes the same point: the difference between a minor incident and a catastrophe is the agent's privilege level.
2. Separate privileged and quarantined components
The privilege separation research proposes a quarantined component that processes untrusted data, kept apart from the privileged component that chooses actions. It goes further and restricts the agent to specific structured data types rather than raw strings. A practical version: have the model that reads an inbound document return a fixed JSON schema (a date, an amount, a category from an enum), validate it in code, and pass only that to the model that can act.
3. Put a human in the loop for high-risk actions
Sending email, deleting records, moving money and changing permissions should require explicit confirmation showing the user exactly what will happen. OpenAI's guidance describes confirmations for high-risk actions as a key safeguard. Design the confirmation screen carefully: a vague "Approve?" prompt invites reflexive taps.
4. Treat model output as untrusted
OWASP lists improper output handling as a separate risk. Do not pass model output directly into a database query, a shell command, or rendered HTML or markdown. Rendering markdown images or links from model output is a known path for leaking data to an attacker's server, so sanitize and restrict what can be rendered.
5. Filter and monitor, as a supporting layer
Input and output classifiers, logging and anomaly alerts are worth the effort because they catch the easy attacks and give you evidence when something goes wrong. Log prompts, retrieved content and tool calls so you can reconstruct an incident.
6. Test adversarially before launch and after every change
Build a regression set of known injection attempts, including indirect ones planted in documents your system would plausibly retrieve. Run it whenever a prompt, model or tool changes.
Prompt versioning is a security control
Prompt versioning is usually pitched as a productivity practice, but it carries real security weight. If anyone can edit a live prompt and have the change take effect immediately, you have an unreviewed change path into your application's behavior.
The AWS guidance on the prompt lifecycle frames prompts as software artifacts that need disciplined lifecycle management. In practice:
- Store prompts in version control with review, like code.
- Pin the version in production code. Some prompt tools default to the latest version, and an edit made in the UI takes effect immediately with no redeploy unless you pin.
- Link each version to evidence. Effective versioning records the prompt text, variables, model settings and evaluation results, so you can explain which instructions produced an output and roll back after a regression.
- Include injection tests in the evaluation set so a prompt change that weakens your defenses fails before release.
For teams that want a practical overview of writing and organizing prompts, this guide to AI prompt engineering covers the day-to-day habits. Pair it with the review discipline above for anything that reaches production.
Prompt design patterns that help
Prompt wording will not stop a determined attacker, but good design reduces accidental failures and makes attacks harder.
- Keep instructions and data visibly separate. Wrap untrusted content in clear delimiters and state that it is data to be analyzed, not instructions to follow.
- Define one narrow task per prompt. A prompt that only classifies a ticket has less room to be hijacked than one that also drafts replies and updates the CRM.
- Specify the output format as a schema, and validate it in code.
- Repeat critical constraints after the untrusted content, not only before it.
- Never put secrets in a prompt. Assume the system prompt can leak.
Treat all of these as friction for the attacker, not guarantees.
A pre-launch checklist for LLM features
| Question | What good looks like |
|---|---|
| What can the model read? | Only data the current user may already see |
| What can it do? | A short, explicit list of tools with narrow parameters |
| Which actions are irreversible? | Gated by user confirmation |
| Where does untrusted text enter? | Mapped: user input, retrieval, uploads, email, web, memory |
| How is output used? | Validated and escaped before it touches other systems |
| How are prompts changed? | Reviewed, versioned, pinned, evaluated |
| How would you know about an attack? | Logged tool calls and alerts on unusual behavior |
Our perspective
Most of the AI features we are asked to look at started as prototypes, often built quickly in an AI app builder or a weekend script. They work in a demo because the demo has one friendly user and no sensitive data. The trouble starts when the same design meets real customers: the model has a broad API key, reads whatever it is given, and can write to systems it never should have touched.
Our view is that prompt injection should be handled in the same conversation as product strategy and architecture, not as a last-minute security patch. Mobile apps make this more pressing, since an LLM feature often sits between a user's device and your backend, and a weak boundary there is hard to fix after release. The questions we ask early are plain ones. What is the narrowest job this feature needs to do? What data does it truly need? What is the worst thing it could do if misled, and can we make that harmless? Answering these changes the design, not just the prompt.
We also push for ongoing care after launch. Models change, prompts drift and new tools get connected, and each change can reopen a hole that was closed. A feature that was safe at release needs evaluation and monitoring to stay that way.
— Brian
Building AI features that hold up in production
If you have an AI prototype that needs to become a secure, production-grade product, our team designs and builds mobile apps, web platforms and applied AI solutions, including LLM integration, AI API integrations, chatbots and backend development. We handle product strategy, UI/UX, engineering, cloud deployment and ongoing support, so security decisions are made alongside the product rather than after it. See our services or review our App & Website Maintenance offering for post-launch care.
FAQ
What is the difference between prompt injection and jailbreaking?
Jailbreaking tries to make a model ignore its safety training, usually through a user's own prompt. Prompt injection targets an application: it smuggles instructions into the model's input so the app behaves in ways its developer did not intend. The two overlap, and a jailbreak can be the payload of an injection.
Can prompt injection be fully prevented?
Not with current techniques. Models have no built-in separation between instructions and data, and filters can be bypassed by adaptive attacks. The practical goal is to limit the impact through least privilege, privilege separation, structured data types and human confirmation of risky actions.
What is the most dangerous kind of prompt injection?
Indirect injection against an agent with access to private data and the ability to act. Incidents such as EchoLeak rated a CVSS score of 9.3 and needed no user interaction, because the assistant read poisoned content while doing another task.
Does prompt versioning help with security?
Yes. Versioned, reviewed and pinned prompts stop unreviewed edits from changing live behavior, and linking versions to evaluation results lets you roll back a regression. It does not stop injection by itself, but it makes your defenses repeatable and auditable.
Sources
- LLM01:2025 Prompt Injection, OWASP Gen AI Security Project
- Better Privilege Separation for Agents by Restricting Data Types, arXiv
- How AI Agents Get Compromised: 6 Attack Patterns, env.dev
- The email nobody opened, Cyril Simonnet
- Managing the generative AI prompt lifecycle, AWS Prescriptive Guidance
- NIST releases new AI attack taxonomy, SC Media
- Prompt injections, OpenAI
- Logic-layer Prompt Control Injection, alphaXiv
- Version prompts, Braintrust documentation
- Prompt versioning guide, PromptLens