Your AI Agent Has Access. What Does It Need to Know?
Three places this question keeps coming up, and a short survey to find more.
Picture an AI assistant helping a customer with a billing dispute. It can see the account, the transaction history, past conversations and a fair amount of personal detail. Partway through, it hands off to another agent to classify the issue and suggest a next step.
What should that handoff contain? The whole history, a redacted transcript, or just the disputed transaction, the issue category and what has already been tried?
In part 1 we looked at why this is a separate decision from access control. In part 2 we showed, on synthetic data, that the preferred representation changes with the recipient's purpose. This post looks at where the question shows up in real systems, and asks where it shows up in yours.
Three places it keeps coming up
Traces sent to observability and evaluation tools
Teams send production traces to tracing and evaluation platforms (LangSmith, Langfuse and similar) to debug agents and track quality. Those traces often carry the full user conversation. The monitoring task usually needs much less: what the agent did, where it went wrong, and whether the issue was resolved.
This is the closest match to what our CIKM 2026 study measured. A triage consumer and an analytics consumer drew on the same events, and each was best served by a different export.
Agent handoffs and MCP tool calls
An orchestrating agent may hold a long context while a particular tool call needs only a small part of it. Passing everything is easy and exposes more than the tool needs. Trimming too aggressively can leave the tool without what it needs to work.
We haven't yet tested Semantic Boundary in production agent workflows, and minimizing what goes into each individual call is an active research area of its own. The shape of the decision is familiar, though: one context, several recipients, each with its own purpose.
Production logs reused as eval sets or training data
Logs collected to serve users often get reused to build evaluation sets or fine-tuning data. That is a new purpose, with different fields and a different audience. It also carries the risk behind our study's first finding: across many entries from the same person, patterns can identify them even after names and obvious identifiers are gone.
What we've heard so far
An earlier exploratory survey of a few practitioners pointed to three recurring concerns: too much information passing downstream, keeping downstream tasks useful after redaction, and being able to inspect what was actually disclosed. That was only a handful of conversations, not a sample, and it has shaped the questions we're asking next. We are looking for many more responses; we'd love your contribution.
Help us map the problem
How do you decide what your agents, tools and downstream systems exchange? And how do you know whether what gets shared is appropriate for its purpose?
We've put together a short survey, about three minutes, on where information sharing gets difficult in your systems and how you handle it today. The answers will help us prioritize our next experiments, evaluation methods and integrations. If one of the three scenarios above looks like yours, or none of them do, we'd like to hear about it.
If you'd rather talk through a real workflow, or have ideas for the open benchmark (new transformations, tasks, threat models or domains), talk to us.
Read further
- Series Part 1: The Right Information, for the Right Reason Part 2: Semantic Boundary, What We Built, Tested, and Learned
- Paper · ACM CIKM 2026 Semantic Boundary: A Framework and Benchmark for Policy-Constrained Semantic Disclosure doi.org/10.1145/3799682.3840076 ↗
-
Open-SBB The open benchmark from the study.Explore Open-SBB ↗
-
Short survey About three minutes. Where sharing gets difficult in your systems.Take the survey ↗
-
Does your system have this problem? gb@nimblenotions.caTalk to us
- ← Back to Semantic Boundary
Semantic Boundary is ongoing research. Our published evaluation uses synthetic data and does not establish production privacy guarantees or regulatory compliance. The billing and agent scenarios are illustrative.