Understand the difference between reading untrusted content and allowing that content to influence a connected business action.
The important boundary is between content and authority
An assistant is asked to summarise a document. That document contains text attempting to redirect the assistant away from the user’s task. This is the core concern behind indirect prompt injection: an external source may contain instructions as well as information.
OWASP describes prompt injection as input that influences a language model’s behaviour in unintended ways. The business impact depends on the surrounding application. An inaccurate summary and an unauthorised outgoing message are different outcomes, even if both begin with misleading input.
Draw the workflow before evaluating the model
Identify what the application reads, what it produces and what it can change. Include retrieved webpages, files, messages, knowledge sources and connected tools. Ask where each source is trusted and what evidence justifies that trust.
For an internal assistant, distinguish a document’s owner from the authority to request an action. Being allowed to read a file should not mean that instructions inside the file can authorise an email, a deletion or a new permission.
Put controls around consequential actions
- Limit the assistant’s access to the information needed for its task.
- Restrict connected tools and accounts to the intended operations.
- Require an appropriate approval before sensitive actions.
- Validate proposed outputs and tool arguments in application logic.
- Keep useful records of inputs, approvals and actions.
OWASP recommends least privilege and human approval for high-risk operations. A prompt telling the model to behave safely is not a substitute for enforceable application controls. Review the code and permissions that determine whether an action actually runs.
Make human review specific enough to be meaningful
An approval screen that only asks “Continue?” may not tell a reviewer what they are authorising. Show the destination, affected records and proposed change in a form the person can understand. Avoid asking someone to approve a long technical trace they cannot reasonably interpret.
Agree which actions need review and who is qualified to approve them. A low-risk draft can follow a different process from an action that changes access or sends confidential information. Make cancellation possible before the action occurs.
Test the boundary in a controlled environment
Use test accounts and non-sensitive example data. Include a document containing an instruction that conflicts with the requested task, then observe whether the application stays within its allowed behaviour. Agree the scope and expected outcome with the system owner before testing.
The useful result is not simply whether the model notices something suspicious. Check whether an unwanted action is blocked, whether the reviewer receives the right information and whether the event is recorded. Retest after changes to tools, permissions or retrieval sources.
Keep the response plan connected to the application
Decide who can disable a connection, suspend a workflow or revoke an application credential. Record the systems that might be affected and how the team will check for unintended actions. Use normal change and incident procedures rather than inventing a separate informal process for AI.
For a first review, bring a workflow diagram, a list of connected systems, permission details and examples of the decisions the assistant may make. These are more useful starting points than a general request to make the model secure.
Take the next step
Turn these questions into a focused discussion with the people responsible for your systems and business processes. Explore our related guidance or talk to a security specialist.