Prompt Injection Is a Role-Design Problem

The June 23 signal was the Role Confusion writeup, highlighted by Simon Willison, which argues that models can misread the authority of text because role boundaries are learned from style as well as system tags. For agents that browse pages, read docs, call tools, or inspect user-generated content, that matters directly: untrusted text can be mistaken for instructions if the product does not preserve the boundary.

Product builders often treat prompt injection as something the model provider will solve underneath the app. That is too weak for real workflows. If your agent reads external data and can also take action, your product needs explicit trust boundaries, scoped tools, logs, and approval points. The user experience should make it clear what is instruction, what is evidence, what is tool output, and what is the agent proposing to do next.

Add a trust-boundary review to any agent workflow. Trigger: the agent reads web pages, tickets, files, emails, comments, or customer text. Context: classify each input as trusted instruction, internal policy, user data, external data, or generated reasoning. Tools: restrict actions by input source. Verifier: injection tests, tool-call review, and approval for destructive actions. Budget: least-privilege credentials and no ambient secrets. Stop condition: untrusted content can inform the answer but cannot authorize action.