Does AI redaction meet GDPR? Redaction vs tokenization

Redaction deletes a value. Tokenization replaces it with a consistent token the model can still reason over. See how each holds up against GDPR and where each falls short.
TL;DR: key takeaways
  • GDPR names pseudonymization as a recommended safeguard, in Article 25 and Article 32, and Recital 28 says pseudonymization reduces risk for data subjects.
  • Redaction deletes the value, so a redacted prompt breaks the model’s ability to reason.
  • Tokenization replaces the value with a consistent token, so the model keeps working.
  • Tokenization is a safeguard, not an exemption.

Short answer: GDPR names pseudonymization as a recommended safeguard, in Article 25 and Article 32, and Recital 28 says pseudonymization reduces risk for data subjects. Redaction deletes the value, so a redacted prompt breaks the model’s ability to reason. Tokenization replaces the value with a consistent token, so the model keeps working. Tokenization is a safeguard, not an exemption.

A familiar pattern: a team writes a regex, strips emails from the prompts, and calls it done. Then the first real use case needs the value back, and the whole thing gets bypassed.

That happens when the choice is framed as redact or don’t. The better question is what the model needs to see, who gets the real value back, and who can prove it afterward. This guide compares redaction with tokenization against the GDPR obligations that matter for AI, and covers where each one falls short. It is part of our wider guide to GDPR-compliant AI.

What is the difference between redaction and tokenization?

Redaction deletes. Tokenization translates. Both hide a personal value from the model, but they leave the model in very different positions.

A redacted prompt reads “[REDACTED] called about [REDACTED].” The model cannot tell who did what, so its ability to reason breaks. A tokenized prompt keeps the grammar, the type, and the consistency. The same person is the same token across the whole document, so the model can track who did what. That is why tokenization works for LLMs where blanket redaction does not.

Tokenization works in four steps. Find the personal data in a prompt, a document, or a dataset. Replace each value with an opaque token that carries the type, keeps the shape, and stays consistent for the same value across the document. Send the tokenized version to the model. Restore the real values only for people and systems that are authorized to see them.

Here is a support ticket before and after:

Before After
Hi, this is Sarah Whitfield, account 4471-8820. My card ending 3310 was charged twice on March 3. Call me at +44 7700 900123. Hi, this is <PER>rk4m2</PER>, account <ACCT>k93f1</ACCT>. My card ending <CARD>p42</CARD> was charged twice on <DATE>d8m1</DATE>. Call me at <PHONE>q27x8</PHONE>.

The model can still summarize, classify, and draft a reply. It knows there is a person, an account, and a duplicate charge. It does not know who. When the agent opens the reply, the tokens are unmasked for that agent because they are authorized to see that customer. When the same ticket feeds an analytics job, the tokens stay masked.

GDPR names pseudonymization as a recommended safeguard in Article 25 and Article 32, and Recital 28 says it reduces risk for data subjects. For AI applications, pseudonymization is not just a safeguard. It is the control that makes the rest of the list easier.

Which GDPR obligations does tokenization help with?

Tokenization touches most of the obligations that an AI application raises. The table maps each one to what tokenization does about it, and where the limit is.

GDPR obligation How tokenization helps
Data minimization (Art. 5) The model never receives the identifier. Minimization is enforced at the prompt, not by policy document
Security of processing (Art. 32) Pseudonymization is named in the article. A leaked prompt log holds tokens, not people
Privacy by design (Art. 25) The control sits in the pipeline, so every new application built on it inherits the protection
Vendor training risk If a vendor retains or trains on your prompts, they retain tokens. The real values never left your boundary
Data transfers (Chapter V) Tokenized data sent to a US API carries far less risk, and the mapping stays in the EU or on your own servers. It is still a transfer of personal data from your side, so keep a transfer mechanism (SCCs or adequacy) and the DPA in place
Right of access and erasure (Art. 15, 17) The token vault gives you a starting index of where a person’s token went. Downstream copies still need integrations and a check that the delete landed
Breach notification (Art. 33, 34) Effective tokenization can lower the risk of a breach, unless the mapping leaks or the remaining text identifies people. Notify the regulator within 72 hours unless the breach is unlikely to pose a risk, notify individuals where risk is high, and document the assessment either way
Accountability (Art. 5.2) Every mask and unmask event is logged. You can show an auditor who saw what, when, and under which policy

The breach row is the one to remember: tokenization lowers risk without removing the duty to assess it.

Special category data is another place where the limit shows. Health, biometrics, religion, politics, sexual orientation, trade union membership, and ethnic origin fall under Article 9 and need an explicit exception on top of the lawful basis. Masking helps, because detection can tokenize a diagnosis or a health condition just like a name, so the model and the tools never see it. But tokenizing does not take the data out of Article 9, since you can still link it back. It shrinks who sees it. It does not change what it is.

Is tokenized data still personal data?

Yes, in your hands, because you hold the key.

The CJEU held in 2025 that pseudonymized data may not be personal data for a recipient who has no realistic means to re-identify people (EDPS v SRB, Case C-413/23 P). That is a case-by-case question. It takes a look at what the recipient holds and what the surrounding text gives away.

Watch out. Treat tokenization as a safeguard that cuts your exposure and supports compliance, not as a way out of your obligations.

The same applies to anything you build with the tokenized output. A tokenized prompt log, a tokenized RAG index, and a tokenized fine-tune set are all pseudonymous data. They are lower risk than the raw versions, and they still need a lawful basis, a retention period, and a delete path.

Why is masking the name not always enough?

GDPR counts data as personal if a person can be singled out from it, directly or by combining fields. A record with no name but with a postcode, a birth year, an employer, and a job title points at one person in most datasets.

This is where simple redaction scripts fail. A pattern that finds emails and phone numbers does not catch the details around them. At scale it gets worse. A RAG index over a million records, or a model trained on them, holds enough of these quasi-identifiers that a determined query can rebuild a profile nobody ever typed in.

Two things follow:

  • Tokenize beyond the obvious types. Dates, locations, employers, and job titles are identifiers in combination, so treat them as such when the dataset is large or the population is small.
  • Watch what the model learns. A model fine-tuned on raw records can be prompted to reproduce a profile even when every name was stripped. The fix is to train on tokenized or synthetic data, not on the raw set with names removed.

Large models memorize. Researchers have pulled training data out of production models with the right prompts, including names, email addresses, and phone numbers. Once personal data is in the weights, there is no simple delete. So the rule for builders is strict: never fine-tune on raw personal data. If you need domain adaptation, fine-tune on masked or synthetic data. A fine-tune set where every name is <PER>x7q2n</PER> and every account number is a consistent token teaches the model the task without handing it the identifiers.

Who gets the real value back?

Masking is only half the system. The other half is deciding who gets the real value back. A good design treats unmasking as a policy decision made at the moment of use.

The support agent assigned to the ticket sees the name. The analyst running a churn report sees <PER>...</PER>. The LLM never receives the original value in this design. Each decision is logged, which is the audit trail Article 5(2) asks for.

This is the piece most teams skip when they build masking themselves. They write a regex, strip emails, and call it done. Then the first real use case needs the value back and the whole thing gets bypassed. Tokenization with controlled restoration is what makes the safeguard survive contact with production.

The same logic applies to agents. Each tool call that carries personal data is a disclosure, so decide per destination which fields stay masked. The agent reasons over tokens. The payments tool gets the real account number because policy allows it. The calendar tool gets <ACCT>...</ACCT>. Log each decision.

Encryption does not replace any of this. Encryption at rest and in transit protects storage and the wire. At the moment of use the model and every tool read clear text, because a model cannot reason over ciphertext. So when someone says “the data is encrypted,” ask where. Tokenization is the control that holds at runtime, because the token is what the model and the tools actually see.

Redaction

Where should the token vault run?

For GDPR the token-to-value mapping is the crown jewels. It is the thing that links pseudonymous records back to identified people. It should sit inside your own infrastructure, in the region the data belongs to.

Protecto Vault runs in your private cloud or on-premises, with an air-gapped option, so the mapping never leaves your boundary while the tokenized text goes wherever the AI needs it to go. The residency trade-offs are in data residency for LLMs.

Tokenization checklist

  • ☐ Only the personal data the task needs goes into the prompt, with identifiers tokenized before they leave your boundary
  • ☐ Residual identification risk checked, including identifying details in the surrounding text
  • ☐ Token vault runs in your infrastructure, in region, under your keys
  • ☐ Detokenization is a policy decision per user and per destination, and every event is logged
  • ☐ Each tool or MCP server an agent calls gets only the fields it needs
  • ☐ Fine-tuning and evaluation sets are tokenized or synthetic wherever possible
  • ☐ Special category data in scope: Article 9 condition identified

FAQs

Does redaction satisfy GDPR?

GDPR names pseudonymization in Article 25 and Article 32, and Recital 28 says it reduces risk for data subjects. Redaction deletes the value, so a redacted prompt breaks the model’s ability to reason, and nothing restores the value for authorized users. Whatever you choose, your wider obligations stay the same.

Is tokenization the same as anonymization?

No. Tokenized data is still personal data in your hands, because you hold the key. The CJEU held in 2025 that pseudonymized data may not be personal data for a recipient with no realistic means to re-identify people, but that is a case-by-case question.

Why can’t I just use regex to strip identifiers?

This is the piece most teams skip when they build masking themselves. They write a regex, strip emails, and call it done. Then the first real use case needs the value back and the whole thing gets bypassed. Masking the name is also not always enough, because identifying details can sit in the surrounding text.

Where to start

Start with one workflow. Write the one-sentence purpose, decide which fields the model needs, tokenize the rest before the prompt leaves your boundary, and make unmasking a logged decision. If you cannot answer who can see the real value and where that is recorded, you have redaction, not a safeguard.

See how Protecto approaches GDPR compliance for AI.

Sources

PII and PHI Identification in Unstructured Text
Free benchmark
PII and PHI Identification in Unstructured Text

A quantitative benchmark by UT Dallas and DataXpert.

Tokenize before the prompt leaves your boundary

See how detection, tokenization, and controlled unmasking fit into your pipeline. Book a demo to talk through a pilot.

About the author

Amar Kanagaraj

Founder and CEO of Protecto

Amar Kanagaraj is the Founder and CEO of Protecto, a company focused on securing enterprise data for LLMs, AI agents, and agentic workflows. He is a second-time entrepreneur with 20+ years of experience across engineering, product, AI, go-to-market, and business leadership. Before Protecto, Amar co-founded FileCloud and helped scale it to over $10M ARR as CMO. Earlier in his career, he worked at Sun Microsystems, Booz & Company, and Microsoft Search & AI. He holds an MBA from Carnegie Mellon University and an MS in Computer Science from Louisiana State University.

Share article

Table of Contents

PII and PHI Identification in Unstructured Text
Free benchmark
PII and PHI Identification in Unstructured Text

Turn these challenges into your next AI advantage.

Talk to a solutions engineer about securing your data privacy, governance, and compliance.

New: run any AI model without exposing real values.
See how it works