Prompt injection

What it is

IBM describes prompt injection as a cyberattack where malicious inputs are disguised as legitimate prompts, manipulating AI systems into leaking data or misbehaving. A direct injection comes from the user. An indirect injection hides in content the model processes, such as a web page or a file.

How it works

1. The model can’t reliably tell trusted instructions from instructions embedded in data.
2. An attacker plants text, such as a request to send data to an outside address.
3. If the agent has tools (email, files, payments), the injected instruction may turn into a real action.

Benefits

Understanding the attack is what makes defenses possible:

• It shapes safer agent design: least privilege, approvals and logging.
• It explains why vendors separate system instructions from user content. Anthropic’s guidance adds screening inputs with a lightweight model and validating inputs.
• It gives security teams a concrete checklist item (OWASP LLM01:2025).

Limits and risks

• There’s no complete fix today. Defenses reduce the risk rather than remove it.
• The riskiest setup is an agent that combines private data, untrusted content and the ability to send data out.
• Filters can be bypassed by rephrasing.
• IonBin doesn’t cite specific named incidents here, because we couldn’t verify any to our standard.