NEW · What your AI agents can access vs. what they actually do with it. Live with ISMGSee how →
What’s the worst thing your agent can do?

What’s the worst thing your agent can do?

Permissions decide what an agent can reach. Data authorization decides what it may do with what it finds.
Pranava Adduri, CTO, 
Co-founder, Bedrock Data

Pranava Adduri

CTO, Co-founder

October 19, 20267 min read
A triangle split into three sections around a dotted gap at its center: every grant can be correct, and the gap is between them

Revised September 22 for clarity, with a new section on the knowledge graph. Part 2 covers handoffs between agents.

An engineering team gave a root-cause agent a routine job: investigate a backend failure and file a bug report detailed enough for the responsible team to reproduce it. The agent traced the failure to an open-source library, built a runnable reproduction from the company's private application, stripped the comments and unrelated code, and posted it to the library's public GitHub issue tracker. The reproduction still contained the company's proprietary business logic and was now public.

Nothing in that sequence was unauthorized. The agent had read access to the application because fixing bugs requires it. It had permission to file issues on external repositories because the team had delegated upstream reporting to it. It did exactly the job it was given. The permissions said what the agent could reach. They did not say what it could do with what it found.

Permissions answer the wrong question

Permissions establish which resources and tools an agent can access. They do not establish whether a particular use of that access is allowed. Reading the application and reporting a dependency defect upstream were both inside the agent's grants. Publishing proprietary logic was not inside the company's rules, and nothing in the workflow connected the two.

This is not a gap a better prompt closes. The agent could identify the right project, write a useful report, and remove the comments, and the pricing algorithm in the reproduction would still be the company's pricing algorithm. The decision about whether it may leave the company depends on what the code is, not on whether the agent was allowed to read it.

Tightening permissions does not fix this case

The usual response is to scope the permissions harder, and it does not work here. The acceptable reproduction and the leaked one leave through the same permission: create an issue on the upstream repository. Revoke it and the agent cannot report the bug. Keep it and the agent can publish the pricing logic. There is no narrower grant between those two, because the difference between them is in the content of the report, not in the resource it targets.

Least privilege still matters. An agent that never held the upstream permission could not have caused this disclosure, and a review of what the job requires is the cheapest control available. But it answers a different question: whether the agent should have the access at all. Once the answer is yes, permissions have said everything they can say.

Decide what the agent may do with what it reads

The decision has to be made on the content of the report, not on the destination, because both reports go to the same tracker through the same API call.

We call this data authorization: before an operation executes, establish what information it carries, which restrictions apply to that information, and who will receive it, then allow or deny the operation. The agent can investigate private code freely. The check is at publication.

Those three questions cannot be answered from the report alone. They need a layer that knows what the enterprise's data is and how the business uses it: an enterprise knowledge graph, which records the data, the identities that own it, where it derives from, where it flows, and who may approve its release. Data authorization draws on that graph. Without it, the service sees text. With it, the service sees the company's pricing algorithm.

Here is where the check sits for the root-cause agent.

Diagram: an agent inside a sandbox files an issue to GitHub, and the request is authorized against the enterprise knowledge graph before it leaves the sandbox

The agent files the issue from inside a sandbox. Before the report crosses the boundary, the sandbox checks it against the knowledge graph: what the code is, where it came from, whether it may leave.

Before the request to create the issue is forwarded, an enforcement point hands the report to an authorization service. The service consults the knowledge graph. Content fingerprinting (matching text against known sources, which survives removed comments and light rewrites) ties the snippet in the report to the private repository the graph records as IP. The graph supplies the rest: the sharing policy on that repository, the fact that a public issue tracker is outside it, and who can approve an exception. The service denies the request. A reproduction built from an approved public example passes through the same call. A denied agent can revise the example and resubmit, or escalate to whoever the company has designated to approve disclosures.

Enforce the decision independently of the agent's account of what it did. An agent's statement that it removed the sensitive parts is a helpful summary and useless evidence.

Three complications the check has to handle

  1. History. Some restrictions depend on what the agent has already read. We see this with our private-equity customers: documents from separate deal rooms cannot be combined, so the service has to evaluate the next read against what the agent has already retrieved. Checking each file on its own cannot show that their combination is prohibited, and blocking an eventual publication is too late. The check runs before the read.
  2. Drift. Data changes under stable permissions. A folder approved for operational documents acquires employee records, and the agent's exposure expands without a single grant changing. The graph has to reflect what the data is now, both when access is reviewed and when a use is authorized.
  3. Coverage. The check has to cover every publication path. An agent that is blocked at an MCP tool and free at a command-line client is not blocked. NVIDIA's OpenShell shows one place the check can run: a sandbox boundary outside the model with control over filesystem and network access, and supervisor middleware that inspects outgoing HTTP requests before forwarding them. That covers requests from tools, scripts and generated code inside the sandbox, provided the deployment routes every publication path through it.

Why this is not already done

Defining permitted use is organizational work, and it has no natural owner. The repository owner knows what the code is. The disclosure policy may assign approval to someone else. Lineage lives in a third system, if anyone recorded it. Connecting them is nobody's backlog item until an agent publishes something, at which point it is everybody's. This is the gap the knowledge graph fills: it holds the data, its owners, its lineage and its approvers in one place, so the answer exists before the agent asks the question. The execution system can supply trustworthy identity and context and enforce a decision. It cannot decide what the company's rules are, and it cannot know what the data means.

Before deploying an agent with sensitive access

  1. Confirm the access is part of the job. If it is not, remove it and stop here. If it is, the remaining steps are where the control lives.
  2. Decide who approves a disclosure, and record it. The check in step 3 has nowhere to escalate until someone owns that decision.
  3. Put a data-authorization check in front of every publication path, and deny paths that cannot be inspected.
  4. Make the check hold when agents hand work to one another. A reporting agent that never read the private repository can still publish what a diagnostic agent handed it. That case is the subject of the next article.

Permissions decide what an agent can reach. Data authorization decides what it may do with what it finds. Both require knowledge of an enterprise’s data, and that knowledge has to live somewhere the authorization service can ask it. An agent with sensitive access needs all three.

At Bedrock Data, we're building Metadata Lake, a knowledge graph of the enterprise's data: what it is, who owns it, where it derives from, which restrictions apply and whose approval is required, available to the authorization service at the moment an agent acts. If you are working through these decisions in your own deployments, we would like to compare notes.

Related Tags:Agentic AIData SecurityMetadata Lake
Share:

Related Resources