Test the boundary before you trust the agent.

Begin with a real task

Imagine an assistant preparing a short project summary. It can read a shared folder and produce a draft for an internal reviewer. It has not been authorized to send those files to anyone else. That is our fictional test environment: a useful assignment with a clear boundary.

Now place an instruction inside a source document asking the assistant to upload the folder to an outside destination. The instruction looks like part of the material the assistant needs to read. The test is whether the system treats that content as evidence to summarize or as authority to change the assignment.

Use synthetic documents and an isolated test destination. A security exercise should not need a real disclosure to reveal whether a boundary works.

Observe the action

The assistant may say that it refused. That statement matters, but it cannot be the only evidence. Did a tool call attempt the upload? Did the surrounding system block it? Did any bytes arrive at the test destination? Those observations answer different questions.

A blocked attempt demonstrates something about the external control. An agent that never attempts the action may demonstrate a different behavior. Neither observation, on its own, establishes broad reliability across tasks. Keep the attempted action, the control decision and the observed effect in separate fields.

Keep the useful work in the test

A system that refuses everything can avoid one kind of failure while becoming unusable. In the same test, ask whether the assistant still produces the authorized project summary, with the evidence needed by its reviewer.

That creates two outcomes to examine together: unauthorized effects and legitimate task completion. Add reviewer effort and recovery time when they matter to the workflow. Avoid reducing the whole exercise to a single pass label that hides which part succeeded.

Test the repair on something new

When a weakness is found, save the original case, the exact system version and the proposed repair. Then evaluate the changed system on fresh cases that were not used to design the fix. Include ordinary work as well as adversarial inputs.

Keep the task, tools and resource allowance comparable. Otherwise, a longer prompt, more computation or an extra review stage can look like an improvement in the agent itself. Each may be useful, but the explanation should match what changed.

Make the evidence portable

A compact record should let another reviewer reconstruct the task, its authority boundary, the input, the attempted operation, the control decision and the actual effect. Include unresolved observations instead of filling them with a confident narrative.

This is an editorial testing pattern connected to my current cybersecurity and governance research agenda. It is not a report of a completed experiment or a claim that a particular product has passed these tests. The public research pages describe the proposed comparisons and measures.

Before the next AI demo, ask: which boundary are we testing, and what evidence would show that useful work stayed inside it?