Runtime Data Security for Agentic AI: 6-Part Series
Part 1: Why Traditional DLP Breaks in Agentic AI
Part 2: Beyond Masking: The Challenge of Safe Data Reveal
Part 3: Authorization Is No Longer Static
Part 4: Why Simple Masking Kills AI Accuracy ← You are here
Part 5: Sensitive Data Is More Than PII (Coming Week 5)
Part 6: From Data Classification to Runtime Data Security for AI (Coming Week 6)
Here is a document going into an AI assistant:
"John introduced Sarah to Michael because Michael had previously worked with Emily on the Delta Bank migration. Sarah later approved Michael's proposal after discussing it with John."
A simple masking system produces:
"[REDACTED] introduced [REDACTED] to [REDACTED] because [REDACTED] had previously worked with [REDACTED] on the[REDACTED] migration. [REDACTED] later approved [REDACTED]'s proposal after discussing it with [REDACTED]."
The information is protected, but the document has become almost impossible for the AI to reason about. It no longer knows who introduced whom, who approved the proposal, or whether the same person appears multiple times. By removing identity, we destroyed the relationships that give the document meaning.
This is the failure mode nobody demos: masking that works as security and fails as engineering. The agent does not crash. It quietly gets worse, guessing around the holes you punched in its context, and you find the problem in production when the answers are subtly wrong. The goal was never to hide data. The goal is to lower risk while keeping the usefulness the system runs on, and most masking gets the first half and quietly drops the second. Accuracy is now a security requirement, because a control that protects data but destroys reasoning will not survive production.
This post is about context-preserving protection: how to protect data while preserving AI reasoning, changing sensitive data so it stays safe and stays useful.
What You’ll Learn
- Why bad masking quietly makes AI inaccurate instead of crashing it
- The three things redaction destroys: format, identity, and meaning
- Format-preserving, consistent, and context-preserving techniques
- Why accuracy is now a security requirement
The Runtime Question
Every decision in this series reduces to the same six questions, evaluated at runtime as data moves between parts of the system:
- Who is asking (the user or caller)
- What data or business context is involved
- Agent is acting (which agent or sub-agent)
- Why it is needed (the task or purpose)
- Where the data is going (the destination)
- When in the workflow (the runtime moment)
The Old Mental Model That Breaks AI
The instinct is “remove the sensitive part.” It comes from the human-reader world, where masked output goes to a person who can work around gaps: a redacted contract is still readable, a blacked-out report still makes sense, because the person fills in the structure from experience. Redaction aims for one thing, that the secret is unreadable, and for a human reader that is enough.
LLMs and tools are not human readers, and this is where the thinking breaks. A model reasons over the patterns in its whole context. Every token carries a signal: the format of a value tells it what kind of thing it is, the steady reuse of a name across turns tells it two mentions are the same person, and the surrounding words tell it how much it matters. When you swap a value for [REDACTED], you do not just hide a secret. You destroy format, identity, and importance all at once. The model loses the ability to check types, to connect mentions, and to weigh things, and it makes up the difference by guessing.
So “remove the sensitive part” aims for unreadability and ignores the thing that actually matters for AI: the protected data still has to carry the same working meaning as the original. Good masking is not about how little the model can read. It is about how much usefulness survives at a given level of risk.
Why this breaks in agentic systems, by what gets destroyed
Bad masking hurts an agent in three different ways, each fixable with a different technique.
It destroys format, so the model loses type and validity
4111 1111 1111 1111 masked to XXXX no longer looks like a card number. Downstream checks break, a tool that expects a 16-digit field rejects it, and the model can no longer tell that “this is a payment instrument.” Format carries the type, and stripping it makes the data unreadable to the very systems meant to handle it.
The fix is format-preserving masking: replace the value with a format-preserving stand-in that maintains the expected structure. 4111 1111 1111 1111 becomes a same-shape token (not a usable card) that still passes a card-shaped check, still flows through code that expects a card, and still tells the model “payment instrument” without exposing the real number. The data behaves like the original everywhere format matters.
It destroys identity, so the model loses the thread
If John Smith becomes [NAME] in one sentence and [REDACTED] in another and Customer in a tool response, the model cannot tell these are the same person. Worse, if every name maps to the same generic token, it cannot tell different people apart. The model loses track of who is who, and an agent that cannot follow people across a conversation, a document set, and a database will mix them up and take wrong actions.
The fix is consistent masking: a given real value always maps to the same stand-in, everywhere. John Smith becomes Person_42 in this prompt, in the retrieved document, in the database row, in the next turn, and in the sub-agent’s context. Consistency keeps the thread: the agent can still reason “Person_42 approved this and Person_42 is the account owner, so the approver is the owner,” without ever seeing the real name. Consistency has to hold across documents, databases, conversations, and agents, which means the map must be steady and shared, not made up fresh at each point.
It destroys importance, so the model loses meaning
This is the trickiest one, and the one format and consistency cannot fix. Sometimes the sensitivity is not a single value but the meaning of a sentence. John Smith gave customer National Bank a 50% discount, approved by his manager as consistent with Tier 1 pricing is sensitive because of the link between a named rep, a named customer, a specific discount, and the pricing-tier strategy behind it. Tokenizing the name to Person_42 and the customer to Org_7 still leaks the structure, and redacting all of it destroys the meaning.
The fix is a context-preserving rewrite: rebuild the content at a lower sensitivity while keeping the working meaning.
The rewrite keeps what the agent needs to act (a notable discount to a strategic customer, approved through proper channels) and drops what it does not (the exact figure, the pricing tier, and the identities). This is no longer find-and-replace. It is writing new text, and it is harder and riskier than tokenization because the rewrite has to be faithful (no invented facts) and reliably lower-sensitivity. It is also the only technique that handles sensitivity that lives in the links between things rather than in a single field.
The tradeoff that governs all three
Every masking choice sits on a risk-versus-usefulness curve, and the detector’s behavior moves you along it. Over-mask (high recall, low precision) and you protect everything and starve the agent, the dumb-agent failure. Under-mask (high precision, low recall) and the agent is sharp and you leak. The job is not to hide as much as possible. It is to find the transform that gives the most usefulness at an acceptable risk for this data and this consumer. A summarizer can take aggressive rewriting. A reconciliation tool needs format-preserving exactness. An analytics job needs consistency above all. There is no single correct masking, which is why masking has to know its downstream consumer.
Beyond PII: business context is where this bites hardest
The format/consistency/rewrite techniques are usually talked about for PII, PHI, and PCI, because those have recognizable shapes. But the data whose meaning is easiest to destroy with bad masking is business-confidential, and it rarely has a shape at all:
- Salary bands and individual pay
- Discount tables and pricing exceptions
- M&A plans, deal multiples, and target names
- Unreleased product roadmap
- Security architecture and incident detail
- Internal investigations and legal strategy
These cases need rewriting more than tokenization, because the sensitivity is in the links. “We plan to acquire Acme at 14x to block a competitor” cannot be protected by masking “Acme” to Org_3. The plan still survives. It needs a rewrite to something like “an acquisition is under consideration for strategic reasons” that lets a planning agent reason about the existence of a deal without exposing its substance. Finding these cases is hard (the topic of an earlier post in this series), and protecting them without making the agent dumb is harder still, because the right transform is the most demanding one.
Real examples
Healthcare. A clinical agent drafting a discharge summary needs the timeline and medications to reason correctly. Redacting dates and dosages produces a dangerous summary. Format-preserving date shifts and consistent patient stand-ins keep the clinical logic intact while removing identity. The agent stays clinically sound.
Banking. A reconciliation tool matching transactions needs card numbers that still validate and still match across systems. Format-preserving, consistent tokens let the same masked card number match across the ledger and the statement. A generic XXXX would break every match and make reconciliation impossible. A comp-analysis agent computing distributions needs real salary bands but not identities. Consistent tokens for employee identity keep the ability to group by department and tenure while removing names. Redacting the bands themselves would defeat the whole analysis.
Customer support. The pricing example. A rewrite that says “a sales rep extended a significant discount to a strategic customer” lets the agent weigh the deal context, while [REDACTED] gave [REDACTED] a [REDACTED] discount leaves it guessing and likely to invent a number.
Enterprise search / RAG. Retrieved chunks have to keep steady entity tokens so the model can connect across documents (“the same vendor shows up in the contract and the incident report”). Inconsistent masking across chunks makes the model treat one entity as several, so it fabricates links or misses real ones.
What to actually build
Default to context-preserving transforms, not removal. Use format-preserving masking so values keep their type and validity, and consistent masking so entities stay trackable across documents, databases, conversations, and agents. Reach for redaction only when no downstream consumer needs any version of the value.
Use rewriting for link-based and business-confidential sensitivity. When the meaning lives in the sentence rather than a field, write a faithful, lower-sensitivity version. Hold the rewrite to a strict bar: keep the working meaning, invent nothing, and lower sensitivity every time.
Make masking consumer-aware. The right transform depends on what happens next. Give a summarizer aggressive rewrites, give a reconciliation tool format-preserving exactness, give an analytics job consistency. One global masking policy will be wrong for most consumers.
Tune the detector as a usefulness dial, not just a safety dial. Recall protects you from leaks. Precision protects the agent from getting dumb. Measure both, and treat over-masking as a real defect, not a safe default.
Keep the map steady and shared. Consistency only works if the same value maps to the same stand-in everywhere, which means a shared, lasting map rather than per-call randomness. This is also what makes safe reveal possible later.
Protecto is one example: format-preserving and consistent tokenization plus context-preserving rewrites, applied with awareness of the downstream consumer, over both PII and business-confidential data. The point is the principle, not the product: protect the meaning, not just the bytes.
