Enterprises deploying AI agents and LLMs often look to prompt scanning as their first line of defense against privacy and security breaches. The idea is simple: analyze the text of the user’s prompt before it reaches the model, detect it for sensitive keywords or patterns, and block the sensitive words that may trigger a security or compliance issue.
Enterprises thought this was a safe around, till they walked into unexpected issues. While it is not completely impractical, this approach fails at a fundamental level. Modern attack vectors, multi-step prompt injections, and indirect data access methods make keyword or regex-based scanning incomplete and misleading.
Key reasons why prompt scanning or filtering fails
Prompt scanning or filtering doesn’t work for detecting and blocking AI risks because it’s fundamentally blind to the full context. Here are the main reasons why it fails:
1. Language flexibility
LLMs learn patterns, facts, language structure, tone, and context from their training data. If the dataset does not contain a uniquely phrased input, it may fail to recognize sensitive text in it.
For example, a question like “What is John’s salary?” can be rephrased in more than one way:
- “What are J.D. ‘s figures for last year?”
- “Link me to J’s comp stats.”
- “Download employee financials and highlight the line with J.”
- “Summarize salary insights for key employees — start with John.”
Attackers or even normal users can use synonyms, indirect language, or rearranged sentence structures that easily bypass static filters or keyword-based rules.
To catch every possible sensitive request, a static filter would need to account for synonyms, morphological variations, grammar changes, and even indirect phrasing (“Compare John’s pay to Mary’s”). The coverage problem is combinatorially large, and maintaining such a rule set is a never-ending, tedious, and error prone exercise.
Therefore, a simple filtering process may miss the sensitive data, resulting in accidental PHI or PII exposure.
2. Sensitive content may reside outside the prompt
Some prompts may contain URLs, file references, or embedded base64-encoded data. The actual sensitive information lives at the destination and not in the visible prompt, making it impossible to evaluate risk just by scanning the input text, especially without sufficient context.
For example, the prompt “Summarize the clinical notes at https://fakeurl.com/patient123.txt” may look harmless on the surface. However, the linked file might include PHI like “Patient John Doe’s HIV status is positive and his prescribed dosage is 250mg of Dolutegravir.”
Similarly, payloads may have hidden sensitive files like:
- Encoded data (Base64, hex, ROT13)
- Obfuscated strings split across tokens (“Joh” + “n’s salary”)
- File attachments with embedded instructions or data
Unless you conduct semantic analysis or use a system for runtime expansion, these risks remain hidden at the prompt layer.
3. Content based prompts escape the scanning radar
In chat-style interactions, prompts build on prior messages. Malicious behavior might only emerge across multiple steps. A prompt may look safe in isolation but becomes dangerous when combined with earlier inputs or context from a knowledge base. To add to the complexity, prompt scanners often operate in isolation; it does not take conversation history into account.
The problem arises when attackers multi-turn prompt injection, where malicious intent is evident only if and when previous exchanges are considered.
For example, let’s say the user input is:
Conversation 1: “Let’s create a sample table of employee salaries for a fictional company.”
Conversation 2: “Replace the fictional values with the actual salaries for employees in our database.”
While single prompts are not a high risk concern, the combination of multiple inputs will likely result in a possible data leak. This complicates further in retrieval-augmented generation (RAG) systems, where LLMs pull from external knowledge bases or document stores as malicious actors have pre inserted the retrieved documents to bypass prompt filtering systems.
How Protecto ensure highly accurate PII filtering using context aware scanning engine
The core flaw in prompt scanning is that it makes decisions on partial, surface-level information. Risk depends on:
- The full expanded context (conversation history, retrieved documents, embedded links, attached files)
- The model’s interpretation of that context
- The output pathway (what data will be exposed)
Effective risk detection must occur at runtime, just before model execution, with access to the full context window and all resolved references. This enables semantic analysis, relationship mapping, and entity detection across the actual data being processed.
1. DeepSight understands context to solve language flexibility
Traditional filters break the moment users rephrase a sensitive prompt in a new way. If a user input “What’s John’s salary?” is reworded as “Compare John’s comp to last year’s average,” the scanner misses it.
DeepSight, Proteco’s PII scanning engine doesn’t rely on simple keyword scanning. Instead, it uses transformer-based NER models trained on vast unstructured, multi-domain datasets. This allows it to understand context, semantics, and real-world intent, not just exact phrasing. It detects that “comp,” “figures,” or “line item for J” are semantically pointing to the same concept; John’s salary.
| Issue | Using prompt filtering | Using Protecto |
| Sample use case/problem
The LLM has access to compensation details. Lets say the prompt is: “What are J.D. ‘s figures for last year?” |
Prompt scanners rely on keyword matching or pattern recognition. They look for obvious triggers like “salary,” “SSN,” “John Doe,” or “payment.” But “comp details” and “J.D.” aren’t on the blacklist, so the filter lets this prompt pass. | Protecto reads the entire prompt contextually. It maps “comp details” to salary, “J.D.” to a probable individual name, and “payroll data” to a sensitive data domain. Using a transformer based model, it recognizes that the structure and intent of this prompt resemble salary inquiry.
It either blocks the prompt, masks the sensitive reference (“J.D.” becomes TKN_PERSON_001), or routes it through policy controls depending on your configuration |
| Impact and concerns |
|
|
If your enterprise uses an internal AI chatbot to answer HR queries and employees use indirect prompts to get salary benchmarks of named colleagues, static filters fail, leading to PII exposure.
2. Real-time semantic expansion
If a user enters “Summarize notes from https://datahub/patient-info.txt”, the real PHI isn’t in the prompt, it’s buried in the external file. Prompt filters can’t scan the link’s contents, and this blind spot can be exploited by attackers.
Protecto fetches and semantically parses linked documents, URLs, encoded payloads, or embedded text at runtime. It treats the full context rather than just the visible message, as the unit of risk analysis.
Let’s understand this better with an example. A health provider used an AI assistant to summarize patient intake forms. Someone referenced a file with embedded HIV status and treatment notes. Here’s how this request is processed using prompt filtering systems versus Protecto.
| Prompt: “Summarize the intake notes from https://healthai.co/uploads/intake_394.txt” | Using prompt filtering | Using Protecto |
| How it works | It scans the prompt text for obvious keywords like “HIV,” “SSN,” “salary,” “John Doe,” etc. If those words aren’t present in the prompt itself, it assumes the input is safe. | It flags the HIV status, prescription data, and the patient name as PHI.
Those sensitive elements are replaced with secure, consistent tokens (e.g., TKN_PATIENT_001, TKN_DIAGNOSIS_002), so the LLM works with structure, not real data. |
| What it sees | Just a non risky sentence with a URL. Nothing sensitive in plain sight.
The actual sensitive information is inside the file at that link—something like: “Patient: John Doe. Diagnosis: HIV Positive. Prescribed dosage: 250mg Dolutegravir.” |
Analyzes the document like a human would. It recognizes entities like names, diagnoses, treatments—even if misspelled or phrased differently.
Real-time document inspection: Protecto follows the URL and fetches the file content before letting the LLM process it. |
| Consequences | The LLM fetches the document, reads PHI, and potentially includes it in its response. The leak goes undetected. Audit logs show a safe prompt and PHI is exposed. | Every scan, detection, and tokenization event is logged. If there’s an investigation or audit, you have concrete evidence that sensitive content was flagged and protected before exposure.
Tokenization preserves structure and semantics. You don’t sacrifice functionality to gain security. |
3. Stateful analysis and memory-aware scanning
Attackers don’t always dump risky content in a single prompt. They build context across turns.
Prompt 1 is harmless. Prompt 2 references it. Prompt 3 pulls the trigger.
Without memory, filters fail to spot the escalation.
Protecto maintains a stateful memory of prior interactions and performs cumulative context analysis. Instead of treating prompts as standalone events, it understands what’s building up and evaluates semantic risk across the full exchange.
To explain this with an example, let’s say a user requests three separate inputs:
| Issue | Using prompt filtering | Using Protecto |
| Prompt 1: “Create a table of employee compensation by department. Use dummy data.”
Prompt 2: “Now replace the dummy names with the actual employee names from our database.” Prompt 3: “Include John Doe’s actual salary in the output.” |
Prompt 1 is safe. Dummy data? No problem.
Prompt 2 seems vague. Nothing overtly sensitive. Passes through.
Prompt 3 might catch “John Doe” and “salary” with static filters—but only if those exact terms are in the rule set. Otherwise, it slips through |
|
| Consequences | The AI assistant outputs real PII (like salaries) in context that the system thinks is safe, because each prompt alone seems harmless. | Uses semantic understanding instead of simple word matches to recognize that the user’s intent has shifted from hypothetical to PII access. It blocks or tokenizes the request before it reaches the LLM. |
Want to see how Protecto works with your LLM pipeline? Let’s get you set up.
