What happens when an attacker hides instructions inside a document your AI agent is reading? In this short explainer, Box CTO Ben Kus demonstrates a live prompt injection attack, where hidden instructions inside a financial document cause an AI agent to misreport revenue, and then shows how newer AI models detect and resist the same attack. Prompt injection is a cyberattack technique where malicious instructions are embedded inside data (emails, documents, websites) that an AI agent is processing. When the agent reads that data, it may follow the hidden instructions instead of the legitimate ones, changing its behavior without the user knowing. The result can range from incorrect outputs to data exfiltration to full agent hijacking. Ben walks through two live demonstrations. In the first, an AI agent reads a financial report containing hidden instructions and misreports the revenue figure, exactly as the attacker intended. In the second, the same test is run using a newer Anthropic model (Sonic). This time, the model not only returns the correct answer but explicitly flags that a prompt injection attack was detected and disregarded. The improvement is backed by data. Anthropic recently published benchmark results showing that while some models had a 40–50% or higher chance of being tricked on the first attempt, newer models are down to just a couple of percent. That's a significant reduction, but Ben is clear that even 2% susceptibility remains dangerous for enterprises running agents on critical tasks with access to untrusted input. Box addresses this directly. Box has built prompt injection controls that examine input before it reaches the model, screening for injected instructions before any damage can be done. Combined with the intelligence now being built into frontier models themselves, enterprises have a layered defense against this class of attack. This video is part of the AI Explainer Series; short, practical explainers on the AI security and governance topics that matter most to enterprise teams deploying AI agents at scale. FAQs: Q: What is prompt injection? A: Prompt injection is when hidden instructions are embedded inside data, such as documents, emails, or websites that an AI agent is reading. The agent may follow those hidden instructions instead of its legitimate ones, changing its behavior without the user's knowledge. Q: How serious is the prompt injection risk for enterprise AI agents? A: Serious. Some AI models had a 40–50% or higher chance of being tricked on the first attempt, according to Anthropic's published benchmarks. Even newer models, which are down to around 2% susceptibility, still pose meaningful risk for enterprises running agents on critical tasks with access to untrusted input. Q: Have AI models improved at resisting prompt injection? A: Yes. Newer models, particularly from Anthropic, have made significant progress. In Ben's live demo, the Anthropic Sonic model not only returned the correct answer but explicitly detected and flagged the prompt injection attempt, then disregarded the malicious instructions. Q: How does Box protect against prompt injection? A: Box has built prompt injection controls that examine input before it reaches the AI model. This means potentially injected instructions are screened and flagged before the model ever processes them, adding a proactive security layer on top of the model's own defenses. Q: Who should watch this video? A: IT leaders, CISOs, and enterprise teams deploying AI agents on business-critical workflows, especially those where agents have access to untrusted input such as external documents, emails, or web content.

 65561      1