Where I put the boundary around an AI agent

After months of building security tools, I kept returning to the same question: what am I prepared to trust the model with?

The work included security monitoring, code-review and web-testing agents, a private legal-research MVP, and systems for agent control and transaction investigation. The tasks were different, but each needed a way to check what the model produced.

A finding starts as a hypothesis

In the code-review agent, a model can point to suspicious code and suggest a failure mode. I still need tools, tests and reproducible evidence before treating that suggestion as a vulnerability.

The web-testing agent has another boundary: the agreed scope. Its requests and actions need limits. A plausible story about an exploit does not replace a working proof inside that scope.

Permissions live outside the model

For the agent security platform, I put a control layer between the model and the tools, APIs and data it can reach. The model can propose an action. Code checks that proposal against the applicable policy before execution.

The model does not get to select its own permissions, decide where protected data may go, or rewrite the policy that constrains it. Those decisions belong to the surrounding system.

Keep enough evidence to revisit the decision

I want a record of the input, the checks, the model’s proposal, the policy decision and the action that actually ran. Without it, a later investigation depends on reconstructing what probably happened.

The same requirement appears in transaction investigations: retain the data sources, rules and relationships behind a conclusion so that an analyst can examine them. In the legal-research MVP, substantive claims need sources and document access must follow the user’s permissions.

I started out trying to automate more of the work. Building these systems made me more precise about which parts to automate, which to constrain with code, and where a person still needs to decide.